Predictors of Achieving Desired Survey Response Rates in Workforce Research

Background: This study examines factors predictive of response rate goals being ‘achieved’ and ‘not achieved’ by county agencies in state child welfare workforce research project. Research about survey methods abound; however, few workforce researchers have analyzed response rates and the corresponding factors that predict those rates. This is of particular importance in workforce studies as extensive research is ongoing to understand what continues to plague human services with undesirable outcomes of high turnover.  Low survey response rates have the potential to result in nonresponse error, leaving out people of different perspectives or economic groups who may be critical to our understanding of these workforce concerns.  In the present study, a 70% response rate from all counties was desired. Extensive procedures were employed by the research staff to achieve the desired response rate, including multiple follow-up calls and e-mails and promotion of survey participation by leadership; however, over half of the counties did not achieve the desired response rate.

Methods: The study is a secondary analysis of a large-scale cross-sectional child welfare workforce study that included a randomized sample from all counties within a state.  A total of 561 participants responded, resulting in a response rate of 56.5%.  Fourteen counties did not achieve the desired response rate and ten did with rates varying from 32% to 86%.

A binomial logistic regression analysis was conducted to test if a county’s work withdrawal (unfavorable behaviors, such as, absenteeism and lateness), caseload, supervisor support, and organizational commitment are predictive of a county ‘achieving’ or ‘not achieving’ the desired survey response rate.

Findings: The overall success rate of the model at correctly predicting county classification is 79.2%, with sensitivity of 80% and specificity of 78.6%. The model is more accurate at predicting counties who have ‘achieved’ than those who have ‘not achieved’ the desired response rate. Analysis of parameter estimates found only work withdrawal is predictive of counties ‘achieving’ or ‘not achieving’ the desired response rate. For each unit increase in counties’ average work withdrawal scores, the odds of having ‘achieved’ the desired survey response rate changed by a factor of 2.05 (95%CI 1.08, 3.88, p<.028).

Implications: The significant work withdrawal findings indicate there could be substantial non-response bias in low response-rate counties attenuating the scores of workforce withdrawal; meaning those counties with a 70% response rate were more reflective of the actual population parameter for work withdrawal.  Conversely the findings may also indicate the counties with more negative work withdrawal behaviors may have been more likely to respond.  The variables, particularly the organizational factors, related to survey response rates in workforce research needs future exploration. Human services workforce research is conducted within an organizational context, often in a single-organization or system. Understanding the unique elements that influence the success or failure of survey response rates can help researchers tailor their design methods prior to implementation and prevent bias due to low response rates. Future research should add methodological experiments to study design to contribute to the larger body of knowledge on this topic.