Leveraging Large Language Models and Public Comments in Critical Qualitative Social Work Research

Social policy research requires engagement with all levels of the policy process, including executive-level policy implementation. Executive agencies publish proposed rules in the Federal Register to implement laws, which initiates a mandated public comment period before finalizing a rule. Public comments can be a rich source of data for social work researchers. Many rules, however, contain thousands of comments, often submitted as part of a mass-commenting campaign (Balla et al., 2022). Efficiently cleaning and analyzing these comments can be an arduous task. The integration of Artificial Intelligence (AI) in social science research offers new methodological opportunities for managing data and analyzing large qualitative datasets (Hayes, 2025). Large Language Models (LLMs) like ChatGPT make AI accessible to researchers without technical backgrounds and can, when used ethically, improve efficiency and rigor in qualitative research. Critical qualitative research relies on contextual and interpretive approaches to interrogate power and status. This process demands a deep engagement with data, often through practices like memoing, close reading, and collaborative processing. Given this, the use of LLMs in these processes can seem antithetical to the ethos of critical qualitative research. This roundtable has three aims: 1) explore the use of public comments as qualitative data for social policy research, 2) demonstrate the use of LLMs in data cleaning and interpretation for large sets of qualitative data as novice AI users, and 3) facilitate a discussion on the ethical integration of AI into critical qualitative research. Aims 1 and 2 will be achieved through a brief presentation of methods used in a critical discourse analysis of public comments regarding a proposed rule from the Administration for Children and Families to place limits on how states can spend their Temporary Assistance for Needy Families (TANF) block grants. The analysis focused on comments related to TANF funding restrictions on pregnancy resource centers (PRCs). The research team used ChatGPT to assist in cleaning and sorting ~7,000 public comments and developing guiding questions to facilitate memoing. Researchers classified and sorted all comments before engaging in deep reading and memoing of all PRC-related comments. The research team read all memos and identified key themes from the memos. The team used ChatGPT to corroborate themes from the memos. Presenters will discuss how to bulk download comments from the Federal Register, manage the volume of comments, and conduct critical analysis of comments. Presenters will also share prompts used with ChatGPT and the resulting questions and analysis. To achieve aim 3, presenters will facilitate and engage in a discussion with participants centered on methods and concerns to ethically use AI in critical qualitative research. Best practices, opportunities for increased rigor, and questions for further investigation will be discussed. References Balla, S., Beck, A. R., Meehan, E., & Prasa, A. (2022). Lost in the flood? Agency responsiveness to mass comment campaigns in administrative rulemaking. Regulation & Governance, 16, 293 - 308. https://doi.org/10.1111/rego.12318 Hayes, A.S. (2025). Conversing with qualitative data: Enhancing qualitative research through large language models (LLMs). International Journal of Qualitative Methods, 24,1 - 19. https://doi.org/10.1177/16094069251322346