Methods: Eligibility requirements included: inclusion of both an implementation strategy and an evidence-based treatment, and a comparison design meeting standards of rigor advanced by the Cochrane EPOC group (2002). CINAHL, Medline, PubMed, PsycINFO, and SocINDEX were searched using variations of: implementation, evidence-based treatments, and mental health. Data extracted included: research design, clinical intervention, implementation strategies, primary theoretical targets of strategies (using Damschroder's [2009] Consolidated Framework for Implementation Research), implementation outcomes, client outcomes, and cost data. The methodological rigor of the studies was assessed using a scale adapted from quality assessment and reporting guidelines (MQRS, SQUIRE, and WIDER recommendations), and outcome attainment was assessed using an index that incorporates methodological rigor and the statistical significance of the findings.
Results: The search strategy yielded 498 articles. Titles and abstracts were reviewed to determine eligibility, and 28 articles met criteria for full-text review. Thirteen studies were included in the final analysis. The majority of studies (92%) evaluated multifaceted implementation strategies (i.e., strategies with multiple components). However, nearly 25% of the studies included interventions that only addressed one theoretical target (e.g., knowledge and beliefs, characteristics of the intervention, etc.), and the vast majority addressed no more than two targets. Six studies reported only implementation (or intermediate) outcomes (e.g., adoption, fidelity), one study reported only clinical outcomes, and six studies reported both. Cost data was reported in only 8% of the studies. Common methodological problems included: poor theoretical justification for use of strategies, inadequate description of strategies, absence of data on dosage and attrition, and lack of reliable and valid outcome measures. Four (31%) of the studies did not demonstrate effectiveness in either implementation or clinical outcome attainment, while 8 (62%) reported statistically significant results on at least one implementation or clinical outcome. The heterogeneity in the types of strategies and outcomes precluded our ability to come to firm conclusions regarding the effectiveness of the implementation strategies; however, these studies suggest that passive approaches are less likely to be effective than multifaceted strategies that address multiple theoretical targets.
Conclusions and Implications: The evidence-base informing the selection of specific implementation strategies is relatively thin despite several rigorous trials that report promising findings. This review can inform future implementation research by presenting some exemplars in an emerging field, and by suggesting ways to improve upon methodology and reporting. Ultimately, this will improve our ability to more precisely specify the mechanisms by which implementation strategies exert their effects.
