A Statewide Mixed Methods Study of a Child Welfare Employee Selection Protocol: Implications for Policy and Practice

Background and Purpose: In response to a continued child welfare (CW) workforce turnover rate of 35%, the state implemented a research-based Employee Selection Protocol (ESP) to improve employee retention. This assessment protocol was designed as a measurement system to better select CW employees with the requisite entry level knowledge, skills, abilities and values (KSAVs) that are considered minimally essential for effective job performance. The research questions framing the study were:

• What is the level of ESP use?
• What is the fidelity of implementation of the ESP?
• What is the perceived effectiveness of the ESP?
• What are job applicants’ depth of understanding of CW work.?

 Method: University IRB approval was granted for this study. This multi-method study employed a data collection design with three categories a) Understanding, b) Use and Fidelity, and c) Effectiveness in the development of both surveys for assessors and new hires as well as in the semi-structured focus group interview guides. Two surveys were administered during the study, one to new job applicants (N=371)) and one to supervisors/administrators (N=155). The surveys used a Likert scale to rate the degree of agreement or disagreement with each survey item. In addition, a representative sample of CW supervisors/administrators (N=125) participated in 17 semi-structured focus group interviews along with 4 groups of newly hired employees (N=44).  Focus group interviews were typically completed by one or two trained data collectors with interviews typically lasting 1 hour.    

Responses to the two surveys were subjected to a series of Principal Components Analyses (PCA) to empirically identify measured variables (latent constructs). Alpha reliability coefficients were computed for each of the four latent constructs. Additionally, comparisons between survey variables and selected demographic groups were completed (e.g., academic degree in SW vs other academic degrees). Focus group data were analyzed using multiple methods (e.g., constant comparative, consensus, and outlier analyses).

Results: Although the ESP is used throughout the state, there was wide variability in the fidelity of the ESP implementation, and the ESP is effective when used as designed. Participants indicated that policy, procedures, and roles of the agency and Human Resources personnel are unclear in the selection and hiring processes. Assessors reported inadequate or lack of training to implement ESP.  New hires found the ESP to be more professional than past selection processes and indicated that the ESP helped them understand and envision themselves in CW work. Study recommendations were provided, many of which the state is implementing.

Conclusions/Implications: The long term implementation of the ESP for selection of the CW workforce would likely result in improved validity, reliability and job-relatedness of employee selection and hiring processes; enhanced standardization of new employee selection processes; the development of procedures for strengthening application screening of new employees who are suited for work in CW; an increase in child welfare employee retention rates; and ultimately, the strengthening of services provided to children and families by employees.