Threats to Online Surveys: Recognizing, Detecting, and Preventing Survey Bots

Online survey research methods are relatively common, particularly as they are low-cost and offer the opportunity to collect data from large, diverse samples relatively quickly. Since the onset of the COVID-19 pandemic and the accompanying social distancing practices to prevent the spread of the disease, online survey research has increased in the field of social work as well as other scientific disciplines. While securing data integrity has been a growing issue with online surveys for over a decade, recent cyber threats to online research, particularly survey bots, have escalated dramatically (Griffin et al., 2021; Pozzar et al., 2020; Storozuk et al., 2020).

Survey bots, also known as automated form fillers, are computer programs that fill out web-based surveys with random responses (Howell, 2019). Survey bots pose a significant threat to data reliability and the development of unbiased scientific evidence (Griffin et al., 2021; Storozuk et al., 2020). Research findings that include responses from survey bots may result in systematic biases through the introduction of non-human participants that do not accurately represent the intended study population and likely provide random responses to survey questions (Chandler et al., 2020). Without the removal of bot respondents, the translation of study findings into intervention and treatment programs may be grounded in biased study findings (Chandler et al., 2020). Moreover, not removing survey bots in research may exhaust federal, foundation, individual, or other research funds when bots are compensated rather than human participants. Additionally, survey bots waste researchers' time and can further decrease community trust in research.

This roundtable discussion will respond to the threat of survey bots by illustrating a variety of strategies utilized by two research teams to recognize, detect, and prevent survey bots. Two research teams will share their research experiences with survey bots and discuss how they recognized, detected, and prevented survey bots. The first research team will present a community-based mixed-methods study about grandparents raising grandchildren during the COVID-19 pandemic in a southern state of the United States. The second research team will present a study that focuses on the experiences of a research team developing a new measurement tool, the Rural Practice Awareness and Skills Scale (RPASS), to assess student and practitioner awareness of rural challenges and skills needed for rural practice that could be used in rural behavioral health recruitment, retention, and educational initiatives.

Based on the lessons learned from these two research teams' experiences, and grounded in existing literature on the topic, the presenters will offer strategies and implications for social work researchers to prevent survey bots and protect online surveys. The strategies include adding open-ended questions, inattentional and/or logic checks, honeypot items, collection of timestamps at the beginning and end of each survey section, asking identical questions at different points in the study, using embedded reCAPTCHAs, checking survey response patterns, and identifying identical participant email addresses.