Creating Organizational Climates to Support Implementation of Evidence-Based Practices for Vulnerable Populations: Results from the Wisdom Type III Hybrid Effectiveness-Implementation Trial

Background and Purpose: The quality and effectiveness of behavioral health services for disadvantaged populations are influenced by organizational leaders and the climates they develop within human service organizations; however, many leaders lack the tools needed to effectively lead implementation of change. The Project WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid effectiveness-implementation trial tested the Leadership and Organizational Change for Implementation (LOCI) strategy on fidelity to, and clinical outcomes of, a digital measurement-based care intervention for youth receiving outpatient psychotherapy. LOCI is a 12-month leadership training, coaching, and planning strategy designed to create organizational climates that support the implementation of new practices in human services. Earlier results from this trial demonstrated that youth served by LOCI clinics experienced significantly higher fidelity to digital measurement-based care as well as improved clinical outcomes. This study aimed to test the contextual mechanism hypothesized to explain LOCI’s effects on fidelity using a multilevel causal mediation approach.

Methods: In this cluster randomized, hybrid Type III effectiveness-implementation trial, 21 outpatient mental health clinics serving youth in three States were randomly assigned to LOCI or an active control condition. In both conditions, clinicians who delivered psychotherapy to youth received training and technical assistance for 12 months to implement an evidence-based digital measurement-based care intervention. In the LOCI condition, executives and first-level leaders participated in strategy meetings, training, and coaching focused on implementation leadership and climate. Implementation and clinical outcomes were assessed for youth-caregiver dyads (N=234) who entered services during the 12-month study period. The primary implementation outcome was fidelity to digital measurement-based care, assessed at the youth level as the proportion of the intervention delivered (0-1), based on electronic meta-data. The focal mediator was change in clinic implementation climate, assessed using aggregated assessments by clinicians at baseline and the conclusion of the 12-month LOCI strategy. Multilevel causal mediation analysis tested whether LOCI’s effects on fidelity to measurement-based care were transmitted through improvement in clinic implementation climate.

Results: Results of the causal mediation analysis replicated prior work in showing that LOCI clinics experienced significantly greater improvement in implementation climate from baseline to 12-months post-baseline, and had significantly higher fidelity to the measurement-based care intervention, equivalent to a 20-percentage point increase relative to control clinics (total effect = .20, p < .05). Most importantly, the analysis indicated LOCI’s effect on fidelity was significantly mediated by improvement in clinic implementation climate (indirect effect = .14, p < .05). Overall, improvement in implementation climate explained 70% of LOCI’s total effect on fidelity to measurement-based care.

Conclusions and Implications: This cluster randomized trial offers experimental evidence that investments in organizational leaders and organizational climate improve the implementation of effective practices for vulnerable populations. Findings illuminate the important role played by leaders of human service organizations in shaping the processes of care for vulnerable populations and demonstrate how investments in leader training and organizational development can support the delivery of effective, equitable care.

Funding: This research was supported by the National Institute of Mental Health (R01MH119127).

Trial registration: clincialtrials.gov (NCT04096274).