Abstract: Is Evidence-Based Practice Sustainment a Process or an Outcome? Implications for Measurement (Society for Social Work and Research 22nd Annual Conference - Achieving Equal Opportunity, Equity, and Justice)

Schedule:

Sunday, January 14, 2018: 8:44 AM

Independence BR B (ML 4) (Marriott Marquis Washington DC)

* noted as presenting author

Lawrence Palinkas, PhD, Professor and Chair, University of Southern California, Los Angeles, CA

Sapna Mendon, MSW, Doctoral Fellow, University of Southern California, Los Angeles, CA

Suzanne Spear, Assistant Professor, California State University, Northridge, Northridge, CA

Juan Villamar, MSEd, Executive Director, Northwestern University, Chicago, IL

C. Hendricks Brown, PhD, Professor, Northwestern University, Chicago, IL

Background

Sustainment of prevention efforts directed at substance use and mental health problems is one of the greatest, yet least understood challenges of implementation science. A large knowledge gap exists regarding the meaning of the term “sustainment” and what factors predict or measure sustainment of effective prevention programs and support systems. Specifically, it is unclear whether sustainment is an outcome of implementation as described by Proctor and colleagues¹, whether it reflects a (final) stage in the process of implementation, or whether it is both process and outcome of implementation.

Materials and methods

In an effort to design and evaluate a Sustainment Measurement System (SMS), we interviewed 45 representatives of 10 grantees within 4 SAMHSA programs (Strategic Prevention Framework – State Initiative Grants, Sober Truth on Preventing Underage Drinking [STOP-Act], Garrett Lee Smith Suicide Prevention Program, and Prevention Practices in Schools). Data collection consisted of a semi-structured interview to identify experiences with implementation and sustainment barriers and facilitators; free list exercise to elicit participant conceptions of the word “sustainment” and what it will take to sustain their programs; and a checklist of Consolidated Framework for Implementation Research (CFIR) elements to identify which are important for sustainment. Lists of sustainment indicators and requirements were then compiled from each data set and compared with one another to see which items appeared on more than one list.

Results

Four sustainment elements were identified by all 3 data sets (ongoing coalitions, collaborations, and networks, infrastructure and capacity to support sustainment; ongoing evaluation of performance and outcomes, and availability of funding and resources) and 5 elements were identified by two of three data sets (community need for program, community buy-in and support, supportive leadership, presence of a champion, and evidence of positive outcomes. All but 2 of the CFIR domain elements were endorsed as important to sustainment by 50% or more of participants; however, not all of the CFIR elements were identified in the other data sources. The final SMS consists of 38 items, including sustainment indicators (n=3); funding and financial support (n=6); responsiveness to community needs and values (n=6); coalitions partnerships and networks (n=8); infrastructure and capacity to support sustainment (n=9); leadership (n=4); monitoring and evaluation (n=1); and program outcomes (n=1). There is some overlap between these items and one or more SAMHSA grantee reporting systems.

Conclusions and Implications. Although sustainment is considered the final phase of implementation, not all features of successful implementation as identified by the CFIR are considered relevant to predicting sustainment. Moreover, the overlap between indicators, requirements and capacity for and indicators and requirements of sustainment suggest that sustainment is both a process and an outcome. However, sustainment as a process represents a dynamic, longitudinal perspective while sustainment as an outcome represents a status, ross-sectional perspective. Moreover, the analysis raises question of when measures of implementation and sustainment should be treated as independent or dependent variables.

Reference.

Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76.