A Machine Learning Analysis of Social Work Job Advertisements in the Post-Pandemic Era

Background/Purpose:

Social work is one of the fastest-growing occupations in the U.S., with 12% projected job growth between 2020-2030 (compared to 8% for all occupations). The COVID-19 pandemic and in many states, recent or planned reforms in the child welfare, health, and behavioral health sectors have resulted in major changes to the nature of social work. Yet, little empirical information exists regarding changing job demand, and/or to the specific qualifications or skills that employers are seeking in their social workers. This study draws on a database of social work job advertisements to address the following questions: 1) What skills and other job qualifications are employers currently seeking in social workers? 2) To what extent do these skills and qualifications vary across fields.

Methods:

We used Python to build a web scraper to extract all job advertisements with the keyword "social work" available on Indeed.com, the most frequently visited job site in the world, in March 2022. The raw HTML of each post was parsed to identify job titles, salary, organization, location, and job description. Duplicates were eliminated, resulting in a final total of 1,086 unique jobs. We then applied natural language processing techniques (NLP), specifically "BERT", to convert the language to vectors. An unsupervised machine learning technique, cluster analysis, was used to categorize job descriptions by social work field. Named-entity recognition (NER), implemented using the Python Spacy package, was used to identify the most in-demand skills or qualifications in each field.

Results:

Jobs were clustered in the following fields: healthcare (48.1%), schools (15.6%), other child and family (13.8%), and behavior therapy (9.2%). Around half (52.7%) of job descriptions mentioned master's degrees (61.3% in healthcare compared to 44.7% in other fields). Over half of job postings (59.2%) also described licensing requirements (e.g., LCSW, LPC, LMFT). Only 5% of posts mentioned telehealth.

Several common top skills are mental health care, case management, and communication. Discharge planning and crisis intervention are particular top skills for social workers in healthcare. In schools, the unique top skills identified included scheduling/planning and social-emotional support.

Conclusions and Implications:

The study identifies healthcare as the largest field in recruiting social workers. Over half of postings were for those with MSWs or an equivalent master’s degree; licensure was also highly desirable. Across different subfields, we can identify common skills (e.g., mental health care) as well as special ones (e.g., discharge planning). These findings provide important information from the market side, which could help schools and students of social work better monitor the environment of their current or even future employment and adjust correspondingly. Furthermore, by collecting historical data like this, we can understand how the demands from each sub-field grow or shrink and explore how the social work field reacts as social issues change. Also, this study shows the possibility of how advanced techniques in data science can efficiently benefit the social work field.