New methods for studying old work

Abstract

Understanding the task content of new jobs is crucial to understanding labor markets. However, structured, task-level data about jobs in the US is nonexistent for the earlier decades of the 20th century. In this thesis, I create a novel dataset that can be used to study new work in 1940. This involves three main contributions. First, I match individual respondents in the 1940 Census to jobs in the 1940 Census Alphabetical Index of Occupations (CAI) using natural language processing (NLP) techniques. This allows us to identify which respondents were working in new jobs. Using the method I developed, I am able to match 85% of respondents in our sample to jobs in the CAI. The second contribution is to match individual respondents in the 1940 Census to jobs in the 1939 Dictionary of Occupational Titles (DOT). Using the method I developed, I am able to match 82% of respondents in our sample to jobs in the DOT. The third contribution of this work is to provide multiple measures of job complexity, skill requirements, and task composition for jobs in 1940. I create these measures using an NLP system that predicts these attributes based on each job's textual description from the 1939 Dictionary of Occupational Titles. I use later editions of the Dictionary of Occupational Titles to train and evaluate the system. The system is able to predict these measures with an accuracy of over 80%, and its predictions generalize well across years.