Society for Social Work and Research 17th Annual Conference: Social Work for a Just Society: Making Visible the Stakes and Stakeholders

January 16 - 20, 2013

State and local child- and family-serving systems are increasingly turning to cost-effective evidence-based programs (EBP) to improve the outcomes they have been getting with “services as usual” (Greenwood, 2008).  While the rationale for using well-tested programs is compelling, numerous challenges have been identified in replicating clinical trials in large, bureaucratic systems (Aarons et al., 2011; Backer, 2000; Bond et al., 2009; Fixsen et al., 2009; Glisson & Schoenwald, 2005).  Accordingly, researchers and practitioners have argued the importance of assessing implementation outcomes, in addition to intervention outcomes, in order to understand the full impact of EBP implementation—and to point to areas that may need improvement to achieve maximum benefit from investment in EBP (Proctor et al., 2009; Proctor et al., 2011).

Evaluation practice has been striving to meet these demands, and has been greatly enhanced by the development and use of the RE-AIM evaluation framework (Glasgow, 2006).  The overall goal of RE-AIM is to improve the sustainable adoption and implementation of effective, generalizable, evidence-based interventions. It defines critical implementation and intervention outcomes and establishes measurement criteria for the critical domains of Reach, Effectiveness, Adoption, Implementation, and Maintenance.  This theoretical framework for evaluation has been use to guide research in diverse settings and topics, resulting in over 180 published articles (Kessler et al., 2012). This symposium adds to this body of work by focusing on the implementation of family-focused interventions in a state juvenile services system.

The three presentations included in this symposium are designed to systematically assess statewide EBP implementation efforts led by the Maryland Department of Juvenile Services using each of the RE-AIM constructs.  Primary research questions include: 

• Adoption: Have state/regional offices allocated a sufficient number of slots to serve the entire target population?
• Reach: What proportion of the target population is actually receiving en EBP?
• Implementation: Are MST and FFT being implemented with fidelity and adherence?
• Effectiveness: What is the impact of participation in MST and FFT on delinquency and social outcomes?
• Maintenance: Are intervention outcomes sustained after discharge?

The first presentation focuses on adoption and reach, the second presentation focuses on implementation, and the third presentation focuses on effectiveness and maintenance. 

The studies here have direct implications for evaluation methodology and the policy and practice of statewide EBP implementation. Taken together, the results of the presentations in this symposium highlight the relevance of RE-AIM as an evaluation framework for statewide initiatives.  Results have immediate implications for Maryland’s resource allocation, referral protocols, and provider and stakeholder confidence in therapist adherence to the models and program outcomes.

Results also have broader implications for evaluation methodology in the social services.  Findings from this session underscore the notion that child and family outcomes alone are not sufficient to understand the impact of programs or policies.  Evaluators should be examining implementation outcomes in addition to intervention outcomes in order to support implementation and quality improvement.  The RE-AIM evaluation framework used in this symposium operationalizes implementation outcomes that should be included in any systematic evaluation.