Evaluation of a Workforce Intervention Designed to Build Resilience and Supervisor Support in Public Child Welfare Organizations

Background and Purpose

Child welfare (CW) staff are exposed to families in crisis, interpersonal violence among adults in households, family members with mental health issues including substance use disorders, as well as children experiencing poverty and various forms of adverse childhood events. Exposure to these events may manifest into symptoms associated with secondary traumatic stress and when compounded with organizational stressors such as high caseloads, can negatively impact the CW workforce. In response to the CW workforce crisis, the Children’s Bureau funded the Quality Improvement Center for Workforce Development (QIC-WD) to build evidence of interventions that enhance workforce well-being, reduce turnover and/or improve child and family outcomes. The purpose of this presentation is to describe the intervention installation and fidelity of the Atlantic Coast Child Welfare Implementation Center (ACCWIC) Coaching Model on supportive supervision and the Resilience Alliance resilience-based intervention in one of the eight sites chosen by the QIC-WD.

Methods

The intervention was installed in phases beginning with a needs assessment, followed by careful choice of interventions, and choice of an implementation science framework. For this study, the Interactive Systems Framework for Dissemination and Implementation (ISF) and the Evidence-Based System for Innovation Support (EBSIS) were chosen to guide implementation. The fidelity evaluation included measures of adherence, dosage, quality from the perspective of participants, as well as participant engagement from the perspective of facilitators, observers and participants. Percentages and scale mean scores were used to assess fidelity constructs.

Results

Fidelity to the ACCWIC supportive supervision intervention was high on all dimensions of dosage, adherence, quality, and participant engagement. Across seven trainings, adherence was 92%, and dosage was at 98%, ratings of training quality by participants all exceeded the 70% mark and observed and self-reported engagement was high. Findings also suggest fidelity to the adapted version of Resilience Alliance across 32 groups for 24 weeks was strong with 84% of activities carried out exactly as proscribed in the Resilience Alliance Manual. 86% of participants attended 75% or more of the sessions with an overall dosage rate of 80.5%. At the mid-point (73%) and end of Resilience Alliance (77%), training evaluation exceeded the goal of 70%, and participant engagement was extremely high. The observation by facilitators indicated more than 97% of participants listened and about 92% were actively engaged in the sessions. Alliance with facilitators was also very high (83%) as was group cohesion (91%); however one facilitator covering two of the 32 groups was significantly less effective on that dimension (64%) and in creating a positive group cohesion (77%).

Discussion

Fidelity was strong for both parts of the intervention suggesting a potential impact important for CW workforce outcomes of engagement in helpful coping strategies, enhancing feelings of perceived social support, intentions to remain at the agency, actual retention, and organizational culture and climate shifts. This study provides support for the usefulness of the ISF and EBSIS frameworks of implementing innovation for very complex and multi-level interventions. Organizations may benefit from lessons learned from real world implementation in child serving systems.