Implementation of an Evidence-Based Practice in a Community Agency: Dissemination, Evaluation, and Lessons Learned

Background: The use of evidence-based practices (EBPs) in social services is gaining momentum as the standard of care. However, many residential settings employ individuals without advanced formal education and training as Direct Service Professionals (DSPs). Typically, these lower-level workers provide the majority of daily care to the more challenged clients, often leading to poorer quality of client care, staff burnout, and high staff turnover rates. The use of EBPs has the potential to mitigate poor client and staff outcomes in such settings when appropriate training can support fidelity to the intervention model.

A promising intervention is Dialectical Behavior Therapy (DBT), an EBP that reduces suicidality, psychiatric hospitalization, and substance use, while improving treatment retention. However, training is required to successfully implement DBT as a therapeutic intervention, and maintenance relies on reinforcement by the program milieu in which it is delivered. Provision of training on EBPs, such as DBT, is important for both quality of client care and workforce development.

This paper presents a case study of a university-agency collaboration in which a DBT training delivery model was refined, and a process evaluation designed to provide a continuous feedback loop on the implementation of the model.

Objectives: The DBT training initiative sought to create a “DBT-informed program culture” through staff training and program-level consultation. The evaluation designed sought to 1) assess individual staff knowledge and retention following DBT training, 2) evaluate the impact on organizational culture, and 3) assess fidelity of DBT service delivery.

Methods: Five residential programs of Resources for Human Development serving individuals with severe mental illness, chemical dependency, and/or forensic involvement in the greater Philadelphia area volunteered to participate in the training initiative from 2013-2015.

The training initiative had three stages of implementation. In Phase One, 3 days of intensive training on DBT principles and skills was provided to DSPs and supervisory staff. In Phase 2, trainers worked with program administrative and supervisory staff to design implementation of DBT services. In Phase 3, DBT services were provided by program staff with titrated consultation.

Staff knowledge retention was measured through administration of a 20 question assessment of principles and skills associated with the four DBT modules: emotional regulation, distress tolerance, interpersonal skills, and mindfulness. The assessment was administered immediately following training, and 3 and 6 months post-training. Organizational culture was assessed using the Organizational Readiness for Change Part D (ORC-D4) (TCU Institute of Behavioral Research, 2009) measure at conclusion of training Phase 2. Information on staff turnover rates maintained by the Human Resources Department were also collected as a program-level measure. Fidelity of DBT service implementation was measured during Phase 3 and 3 months post-completion using the DBT Skills Group Observation tool (2009, DBT Center of Michigan) and client completion of the Session Rating Scales (SRS v3.0; Miller & Duncan, 2000).

Conclusions: The collaboration led to clearer conceptualization of individual and organizational factors of the delivery model, and its intended impact on services. This case study illustrates one method of sustainable EBP implementation and a low-cost evaluation design.