380P
The National Housing Trust Fund Act of 2007: Using Supervised Machine Learning to Test Policy Process Theory in a Social Welfare Context
Housing comprises the largest household expenditure for many American families, and can seriously impact a household's financial capacity. Further, housing policy has often been used as an anti-poverty strategy in United States policy making. Long-term issues in housing affordability, and the national foreclosure crisis, prompted the federal government to increase funding for affordable housing initiatives with programs like the National Housing Trust Fund (NHTF) in 2007. Despite strong bipartisan support at its adoption (the House passed the law authorizing the program by a vote of 264-148), Congress has yet to appropriate the funds to enact this law. In attempting to understand the circumstances for the act’s passage but paradoxical lack of appropriation, a technical problem emerged. Data were comprised of over 600 hearings containing thousands of pages of text: traditional methods of content analysis would require a large amount of resources to complete the project in a reasonable time frame. The current paper tests an innovative method of content analysis to address this issue: computer assisted content analysis (also known as automated content analysis) using supervised machine learning. Results indicate that supervised machine learning is a useful method to test policy process theory in a social welfare context, but more testing is needed to ascertain the right algorithm mix to capture more nuanced code categories.
Methods
Hearing testimony from 1997-2007 from the Congressional Committee on Financial Services was analyzed using content analysis (Krippendorf 2013). Transcripts and written testimony for hearings held by the full or housing subcommittee were analyzed using Python 2.7 software. Supervised machine learning (Grimmer & Stewart 2013) was then used to code the text into theoretically relevant categories and tested for frequency of occurrence and statistically significant relationships. Two algorithms were trained: support vector machine (SVM) and maximum entropy (MaxEntropy).
Results
Analysis indicates that automated content analysis (ACA) appears to provide a reasonable alternative to traditional methods of content analysis, particularly when used to assess the applicability of broader theoretical concepts. Supervised machine learning resulted in precise (over 75%) coding of central theoretical concepts. Two limitations arose: (1) coding concepts that rely on nuances of communication may require more robust algorithms and (2) use of this method requires the ability to reasonably assume that the validation experiment is compatible with the testing experiment.
Conclusions and Implications
While housing remains at the heart of anti-poverty policy and advocacy, social welfare scholars in the United States have moved away from studying its development. One explanation may be the shift in social work away from policy practice (see Figueira-McDonough (1993); Domanski (1998); Weiss-Gal (2013)). Another explanation may be the dearth of funding options for social welfare policy research. Computer assisted content analysis may provide an effective, innovative way to reinvigorate policy research within the social welfare field by minimizing the amount of time and resources needed to complete research projects.