Session: Threats to Online Surveys: Recognizing, Detecting, and Preventing Survey Bots (Society for Social Work and Research 27th Annual Conference - Social Work Science and Complex Problems: Battling Inequities + Building Solutions)

All in-person and virtual presentations are in Mountain Standard Time Zone (MST).

SSWR 2023 Poster Gallery: as a registered in-person and virtual attendee, you have access to the virtual Poster Gallery which includes only the posters that elected to present virtually. The rest of the posters are presented in-person in the Poster/Exhibit Hall located in Phoenix A/B, 3rd floor. The access to the Poster Gallery will be available via the virtual conference platform the week of January 9. You will receive an email with instructions how to access the virtual conference platform.

230 Threats to Online Surveys: Recognizing, Detecting, and Preventing Survey Bots

Saturday, January 14, 2023: 9:45 AM-11:15 AM
Valley of the Sun D, 2nd Level (Sheraton Phoenix Downtown)
Cluster: Research Design and Measurement
Yanfeng Xu, PhD, University of South Carolina
Yanfeng Xu, PhD, University of South Carolina, Sarah Pace, MSW, University of South Carolina, Jaeseung Kim, PhD, University of South Carolina, Aidyn Iachini, PhD, MSW, LSW, University of South Carolina and Melissa Simone, PhD, University of Minnesota-Twin Cities
Online survey research methods are relatively common, particularly as they are low-cost and offer the opportunity to collect data from large, diverse samples relatively quickly. Since the onset of the COVID-19 pandemic and the accompanying social distancing practices to prevent the spread of the disease, online survey research has increased in the field of social work as well as other scientific disciplines. While securing data integrity has been a growing issue with online surveys for over a decade, recent cyber threats to online research, particularly survey bots, have escalated dramatically (Griffin et al., 2021; Pozzar et al., 2020; Storozuk et al., 2020).

Survey bots, also known as automated form fillers, are computer programs that fill out web-based surveys with random responses (Howell, 2019). Survey bots pose a significant threat to data reliability and the development of unbiased scientific evidence (Griffin et al., 2021; Storozuk et al., 2020). Research findings that include responses from survey bots may result in systematic biases through the introduction of non-human participants that do not accurately represent the intended study population and likely provide random responses to survey questions (Chandler et al., 2020). Without the removal of bot respondents, the translation of study findings into intervention and treatment programs may be grounded in biased study findings (Chandler et al., 2020). Moreover, not removing survey bots in research may exhaust federal, foundation, individual, or other research funds when bots are compensated rather than human participants. Additionally, survey bots waste researchers' time and can further decrease community trust in research.

This roundtable discussion will respond to the threat of survey bots by illustrating a variety of strategies utilized by two research teams to recognize, detect, and prevent survey bots. Two research teams will share their research experiences with survey bots and discuss how they recognized, detected, and prevented survey bots. The first research team will present a community-based mixed-methods study about grandparents raising grandchildren during the COVID-19 pandemic in a southern state of the United States. The second research team will present a study that focuses on the experiences of a research team developing a new measurement tool, the Rural Practice Awareness and Skills Scale (RPASS), to assess student and practitioner awareness of rural challenges and skills needed for rural practice that could be used in rural behavioral health recruitment, retention, and educational initiatives.

Based on the lessons learned from these two research teams' experiences, and grounded in existing literature on the topic, the presenters will offer strategies and implications for social work researchers to prevent survey bots and protect online surveys. The strategies include adding open-ended questions, inattentional and/or logic checks, honeypot items, collection of timestamps at the beginning and end of each survey section, asking identical questions at different points in the study, using embedded reCAPTCHAs, checking survey response patterns, and identifying identical participant email addresses.

See more of: Roundtables