An Evaluation of Advanced Certification Training for Child Welfare Professionals

Background: Improving child and family outcomes requires a thriving child welfare workforce, however, longstanding, widespread issues with turnover make meaningful change unlikely. A longitudinal study of Florida’s child welfare workforce (2015-2020) found 81% of newly hired frontline workers left their agency within 3.5 years, with most departing within the first 18 months citing job responsibilities and the agency environment. In response, the Florida Legislature enacted legislation aimed at supporting a robust workforce that includes providing university-led professional certifications for child welfare professionals. Led by the Florida Institute for Child Welfare, the first advanced certification on trauma and resilience (STARS) was piloted. This paper presents results from the summative evaluation.

Methods: Surveys, post-course semi-structured interviews, and focus groups with participants were used in the evaluation process. The participants completed pre-post-training and follow-up (3- and 6-months) surveys. Also, 12 Interviews and 3 focus groups were conducted. Guided by Kirkpatrick’s (1985) model for evaluating training programs, the summative evaluation focused on the first three outcome levels – their reaction (or satisfaction), learning (knowledge, attitudes), and initial behavioral changes (changes in professional practices). Participant reactions were assessed using brief feedback surveys following each module, learning was evaluated based on quiz performance. Changes between pre- and post-training and follow-up surveys provided data on changes in professional practices by incorporating measures on individual-level workplace outcomes of self-efficacy, trauma-informed care attitudes, resiliency, quality of life, job satisfaction, and intent to remain in the position.

Results: 27 (90%) participants completed the training. The participants showed high satisfaction with training relevance to their job responsibilities and delivery methods (a combination of asynchronous and live sessions) and they performed well on each post-session/module quizzes. Comparison of pre-and post-training and follow-up (3- and 6-months) surveys showed promising preliminary trends. Mean compassion satisfaction significantly increased between pre and post-test by .11 (p=.03) and maintained growth through 3-month and 6-month follow-up. While not statistically significant, participant means also improved on the Secondary Traumatic Stress (-.60)) and ARTIC-10 (.40) measures between pre and post-test that maintained through both follow-ups. During the interviews and focus groups, participants also discussed how the course materials boosted their empathy for colleagues and provided skills and helpful resources they were already using or planned to use.

Conclusion: Preliminary evidence suggests supportive and skill-enhancing programs like STARS contribute to the cultivation of a thriving child welfare workforce and may ultimately benefit the well-being of children and families. A significant reduction in participants' intent to leave their jobs and the increase in compassion satisfaction are particularly meaningful given the high turnover among child welfare professionals. Despite several limitations of the pilot such as the small sample size of the participants (n=30) and 41% response rate (3-month and 29% at 6-month follow-ups), findings from the evaluation of the STARS advanced certification program highlight the potential to address the longstanding issue of turnover within the child welfare workforce. Ongoing efforts to integrate evidence-based practices, incorporate participant feedback, and adapt to evolving workforce needs will be essential for optimizing the impact of such initiatives.