Factors Affecting Fidelity in a Cross-Systems Intervention for Child Welfare and Substance Use Treatment: A Coincidence Analysis

Background and Purpose

In the U.S., many families involved with the child welfare system are impacted by parental substance use. Effective cross-systems collaboration between child welfare and substance use treatment systems can improve caregiver access to treatment but is challenging to achieve. Sobriety Treatment and Recovery Teams (START) is an evidence-based, cross-systems intervention for families with co-occurring child maltreatment and substance use focused on expediting caregiver access and engagement in substance use treatment, among other goals. When implemented as intended, START can increase likelihood of family unification, and reduce subsequent maltreatment risks (e.g., Hall et al., 2021). The current study draws on data from the statewide implementation of START in Ohio to examine contextual and organizational factors associated with higher implementation fidelity, i.e., whether START was implemented as intended.

Methods

Data for the study are drawn from the Ohio START evaluation and include key informant interviews, child welfare agency contracts with addiction treatment providers, frontline worker surveys, administrative data on START fidelity, and secondary data on local system context. In Ohio, child welfare services are county administered; thus, our unit of analysis was the county child welfare agency. Our study included 17 counties in implementing START and working toward model fidelity. Our measure of implementation fidelity focused on: (a) percentage of START cases that received any substance use treatment and (b) that completed four treatment sessions. We also assessed timeliness of service receipt (e.g., per START protocol, treatment within four days of behavioral health assessment, four treatment sessions within 12 days of assessment). Counties were considered high fidelity if they were above statewide averages in the percentage of cases receiving treatment and in timeliness of service receipt. We used a configurational comparative method (coincidence analysis) to identify factors that differentiated counties with high vs. low implementation fidelity. Analyses were run using appropriate functions in the R -cna- package; final model selection was based on 100% consistency, highest coverage, and no model ambiguity.

Results

On average, 58% of START participants in each county completed at least one substance use treatment session (55% of these were timely). Approximately 47% of START participants completed at least four substance use treatment sessions, and 43% of these were timely. Coincidence analysis results identified that either high collaboration (measured via Wilder Collaboration Factors Inventory) or being in a county where services were administered by a single-county behavioral health board was sufficient for high fidelity. Conversely, being in a county where behavioral health services were administered by a multi-county board that did not help connect child welfare agencies to behavioral health providers for START was sufficient for low fidelity.

Conclusion and Implications

Although START includes multiple intervention components intended to facilitate caregiver access to substance use treatment, implementation fidelity is contingent on the quality of collaboration between child welfare and substance use treatment providers. Third parties such as regional behavioral health boards can also help facilitate fidelity by helping broker relationships between child welfare agencies and treatment providers. Our findings can inform regional administration and management practice.