Use of Chatgpt for Qualitative Data Analysis: Implication for Social Work Research

Background. Various computer-assisted technologies for transcription and data analysis have been widely adopted in social work qualitative research. ChatGPT, an artificial intelligence (AI)-driven chatbot stands out as a novel tool with its capability to process human languages. This study aims to explore the usability of ChatGPT as a research tool for qualitative data analysis and provide implications for social work research. Methods. We randomly selected 30 posts from a larger dataset of 882 posts written in Korean about dementia. The initial data were drawn from the largest online community among Korean Americans to explore discourses related to dementia, with the posts made between 2015 and 2023. Human coding was initially conducted by two researchers independently reviewing and coding the posts. Subsequently, the same two researchers independently conducted ChatGPT coding by inserting the same command, 'provide the main themes in English,' for each post in Korean using ChatGPT 3.5. Lastly, we conducted an assessment of ChatGPT as a qualitative data analysis tool by comparing and contrasting the codings between human and ChatGPT using the following criteria: coding quality, linguistic sensitivity, and ethical aspects. Results. First, ChatGPT’s revealed its limitation in contextual understanding, which impacted its coding quality. Similar themes were identified between human- and ChatGPT-coding on detailed posts, but ChatGPT provided irrelevant coding on very simple or nuanced posts. Also, there were some inconsistencies in ChatGPT-coding for the same posts across researchers. Second, ChatGPT demonstrated its high linguistic sensitivity, being capable of directly identifying main themes in English from Korean posts, even understanding Korean linguistic typos or grammar errors. Third, a wide range of ethical concerns arose related to data privacy and confidentiality. Due to the use of public data, it was not applicable for the present study. However, when considering the nature of social work research particularly its focus on social justice and empowerment for vulnerable population, and the extensive information within qualitative data sets, this ethical concern could create greater harms on study participants than anticipated because ChatGPT can store the data inserted and utilize it for other purposes. Conclusion. Social work researchers can benefit from using ChatGPT, especially when working with language minorities, because of its capability for quality translations. However, due to its lack of contextual understanding, ChatGPT may be used only as a supplemental tool. For example, considering its capability to deal with large datasets quickly, social work researchers may use it to develop an overall understanding of a dataset to create an initial coding scheme. Most importantly, there is an urgent need to address ethical concerns. Given the growing utilization and potential of ChatGPT, active discussion and discourse surrounding its usability and practicality in social work research are needed. Also, clear and specific ethical guidelines need to be developed promptly at institutional levels. Our study provides a preliminary but crucial and timely perspective on the use of ChatGPT in social work research. Further studies will be needed to leverage the rapidly developing AI technologies to advance social work research.