Benchmarks for One-Group Outcome Evaluations of Research Supported Treatments

Background and Purpose. Research supported treatments (RSTs) have been empirically supported in randomized clinical trials (RCTs) in which the RST is provided under relatively ideal service provision conditions. Real world social work settings seeking to implement those RSTs typically cannot match those conditions due to having to provide them under more problematic service provision realities, more heterogeneous client problems, more culturally diverse clientele, and limited resources for obtaining training and supervision in the RST.  Consequently, the RST commonly must be adapted to fit the setting, and evaluations are needed to assess whether the RST is achieving satisfactory outcomes in the setting. However, real world practice settings rarely have the resources or proclivity for conducting outcome evaluations with unbiased control groups. This presentation will provide a scientific rationale and a framework for helping agencies utilize the results of their one-group evaluations to guide practice decisions about whether and how to continue to provide the adapted RST or to replace it with a different approach. The rationale will explain why –in light of the extensive RCT support for RSTs -- one-group outcome evaluations in those settings can merit guiding practice decisions despite the absence of a control group.

Methods and Results. The presentation also will provide the methodology and results of a meta-analysis of one-group effect sizes (calculating separate pre-to-post effect sizes for experimental and control groups) in published RCTs that have provided the empirical support for the following three RSTs for treating traumatic stress: prolonged exposure therapy (PE), cognitive processing therapy (CP), and eye movement desensitization and reprocessing (EMDR). These results will include descriptive statistics on the central tendency and dispersion of one-group effect sizes in RCTs for each of the above three RSTs, as well as for waitlist control groups and for treatment as usual control groups. The results also will be presented separately regarding whether or not an intent-to-treat analysis was conducted.

Conclusions and Implications. The presentation will provide a framework for conducting similar meta-analyses that can provide benchmark effect sizes for other target problems to which agencies can compare their one-group effect sizes as a basis for guiding practice decisions about their adaptions of RSTs. The results of these studies can also have utility for future research efforts to assess what service provision conditions are associated with effect sizes that resemble the RST benchmarks derived from RCTs.