Abstract: Growing Pains of Virtual Data Integrity in Measuring Discrimination: Lessons Learned from an Intersectional Mixed Methods Study with 2SLGBTQ+ People (Society for Social Work and Research 27th Annual Conference - Social Work Science and Complex Problems: Battling Inequities + Building Solutions)

All in-person and virtual presentations are in Mountain Standard Time Zone (MST).

SSWR 2023 Poster Gallery: as a registered in-person and virtual attendee, you have access to the virtual Poster Gallery which includes only the posters that elected to present virtually. The rest of the posters are presented in-person in the Poster/Exhibit Hall located in Phoenix A/B, 3rd floor. The access to the Poster Gallery will be available via the virtual conference platform the week of January 9. You will receive an email with instructions how to access the virtual conference platform.

Growing Pains of Virtual Data Integrity in Measuring Discrimination: Lessons Learned from an Intersectional Mixed Methods Study with 2SLGBTQ+ People

Friday, January 13, 2023
Maryvale A, 2nd Level (Sheraton Phoenix Downtown)
* noted as presenting author
Tin Vo, MPH, Doctoral Candidate, Wilfrid Laurier University, Kitchener, ON, Canada
Background and Purpose: There has been an increase in intersectional 2SLGBTQ+ research addressing the experiences of different subgroups (e.g., racialized trans people), including studies using online surveys. Online surveys can help to recruit large diverse samples and engage hard-to-reach groups (e.g., those not out). Despite their benefits, ensuring data integrity in online surveys is challenging, especially given the risk of fraudulent respondents. They can also be limited in terms of integrating an intersectional lens, including assessing discrimination in an intersectional way and operationalizing McCall’s (2005) intercategorical complexity. Drawing on the implementation of the Wellbeing & Inclusion in Rainbow Leisure Study (WIRLS), this presentation reports on these issues and offers some solutions for researchers to consider in their studies.

Methods: WIRLS used a sequential explanatory mixed-methods design to examine 2SLGBTQ+ people’s experiences and outcomes in 2SLGBTQ+ leisure spaces (e.g., gay sport leagues, trans chatrooms). Participants from Canada and the US were recruited via social media and 2SLGBTQ+ organizations using purposive and quota sampling (e.g., 100 trans) for an online survey ($10 incentive; n=548; 49.1% Canadian; 55.5% trans; 34.1% racialized; 49.3% disabled) and semi-structured interviews ($50 incentive; n=22; 36.4% Canadian; 63.6% trans; 68.2% racialized; 50.0% disabled). Assessing intersectional discrimination was a priority; Scheim & Bauer’s (2019) Intersectional Discrimination Index focusing primarily on direct/personal discriminatory experiences was adapted for the 2SLGBTQ+ leisure context with additional items developed in collaboration with WIRLS’ advisory committee. To understand intersectional discrimination among various subgroups, an intercategorical analysis was conducted using two-way and three-way interaction terms with gender, race, and disability.

The survey included data integrity strategies (e.g., multiple choice and open-text quality control questions, Captcha). However, additional tactics were needed given the suspicious responses upon initial launch; a separate screening survey and metadata functions in Qualtrics were implemented, and every screening survey response and the metadata were reviewed to judge the validity of submissions.

Results: All 167 responses submitted upon the survey’s initial launch were deemed to be fraudulent or bots (e.g., IP address outside North America, nonsense open-text responses). With the additional data integrity strategies, approximately 6% of responses were classified as fraudulent. Scores on the intersectional discrimination scale were relatively low (M = 0.36, SD = 0.50; theoretical range of 0-3), but interviewees, including those reporting little discrimination on the survey shared many experiences, including indirect/ambient discrimination targeting other members. Despite the relatively large sample, group size issues arose when examining three-way intersections of identities (gender X racialized X disability) to reflect the intercategorical complexity. Qualitative analysis identified gender identity/expression, race, and disability and associated oppressive systems as influential in shaping participants’ experiences in 2SLGBTQ+ leisure spaces.

Conclusions and Implications: The additional strategies employed in the survey minimized fraudulent respondents and bots, thus promoted data integrity. Though the Intersectional Discrimination Index is useful, it is important to address indirect/ambient discrimination to holistically understand discrimination. Given the inability to interact various positionalities in survey research, mixed-methods research can address this and provide insights into the complexity of intersectional discrimination.