EBP Implementation across Systems: Responding to Organizational and Workforce Challenges in Child Welfare and Mental Health

Background: Many children and youth involved with child welfare services experience substantial mental health symptoms; however, there is a lack of coordinated child welfare and mental health services. The National Center for Evidence-Based Practice in Child Welfare developed an integrated, sustainable, cross-systems workforce competency model, Partnering for Success (PfS), designed to improve mental health outcomes for child welfare-involved children and youth. The “partnering” aspect of PfS intends to overcome the long-standing lack of effective policy, program, and practice coordination between the child welfare and mental health systems. PfS uses a components-based version of cognitive behavioral therapy (CBT+ for anxiety, behavior problems, depression and trauma) as its core evidence-based intervention. The current study examines the diverse array of implementation activities and workforce practice enhancements across both service systems and the impact on child mental health outcomes.

Methods: PfS utilized a High Fidelity Performance Indicator (HFPI) framework (i.e., Readiness; Adherence; Quality; Reach; Dosage; and, Participant Responsiveness) to examine implementation across 4 nationwide sites representing 7 unique cohorts. Readiness was assessed through an organizational assessment delivered prior to PfS participation. Adherence and Quality were measured through participation and skill assessment across a variety of activities (e.g., knowledge tests, practicum participation and Capstone presentations). Assessment of Reach occurred through the evaluation of a series of Leadership and Implementation Team meetings (including senior executives, supervisors, and direct service professionals). Dosage was measured through the characteristics and number of CBT+ sessions. Participant Responsiveness was assessed through changes in child mental health outcomes.

Findings: The readiness survey revealed some uncertainty across all sites. However, as a whole, sites demonstrated a degree of readiness that indicated proceeding with PfS. Adherence (completing all components - training, on-line modules, consultation calls, capstones) was high across sites for child welfare professionals (90%), but much lower (50%) for mental health professionals. Quality was achieved, as both child welfare and mental health professionals who completed PfS activities showed significant improvement in knowledge acquisition at each PfS site, with 82% of child welfare professionals and 95% of mental health professionals reaching the highest rating (i.e., successful). Leadership session meeting evaluations indicated an enhanced ability to monitor the implementation of PfS and promote provider buy-in. Children and youth referred to receive CBT+ received a range of 9-15 CBT+ sessions (dosage). Data indicated that children and youth receiving CBT+ consistently moved from the clinical to non-clinical range across treatment targets, or moved from higher to lower levels of clinically indicated findings (participant responsiveness).  

Discussion: Promoting change across systems, and being able to assess effects, requires the ongoing commitment of a highly functioning leadership and implementation team. Findings suggest that this model is feasible, and can potentially be replicated through the utilization of a structured series of capacity building activities, including assessments, consultation guides, a tailored curriculum, and practice applications to support evidence-based practices. Together, these supports can help to sustain and scale the model. Further research will help identify predictive factors associated with the success of the model.