Variance Decomposition Approach to Studying the Relative Importance of Relationships in Program Outcome

Background and purpose: Increasingly intervention research in social work seeks to identify the key elements or ingredients that drive the effects of an intervention. Research on comprehensive services in substance abuse treatment (SAT), i.e., services that integrate health and social services with substance abuse interventions, have identified three specific service components that, when part of the treatment model, result in improvements in both intermediate and ultimate outcomes. Across a series of studies, these components include access services (transportation and child care), matched services (when clients receive specific services they say they need) and client-provider relationship (a client rating of the alliance or connection with their service provider). Although substantial evidence indicates each of these components is associated with or predictive of positive outcomes in substance abuse treatment, there is little evidence of the relative importance of each. In general intervention research has been preoccupied with testing the overall impact of treatment without documenting the components or elements of treatment most predictive of client outcomes. Very little research has attempted to assess the relative importance of a set of service components for accounting for variance in outcomes.

Methods: Data for this analysis were collected from the 1992-97 National Treatment Improvement Evaluation Study (NTIES), a prospective multilevel cohort study of SAT programs and clients. Organizational-level data was collected from an administrator survey and client-level data was collected from a client survey administered at treatment entry, discharge and 12-months post-discharge. This data set is unusual for its data collection (1) at both organizational- and client-levels, (2) over-time, at pre-, post- and 12-month follow-up, and (3) of process as well as outcome variables including a measure of client-provider relationship. The analytic sample consisted of 59 service delivery units and 3,027 clients:1,123 women and 2,019 men, 1812 Blacks, 486 Latinos and 844 Whites. To assess the relative importance of the service ingredients, a linear regression variance decomposition approach was used while considering the nested nature of NTIES data. Specifically, hierarchical model comparison was used to measure the contribution of key service ingredients to post-treatment drug use reduction. First, a model Chi-square of a base model that included control variables as well as prior drug use was obtained. Then, each service ingredient was sequentially added to the model in the following order: access, substance abuse counseling, need-service match ratio and client-provider relationship. A significant increase in the model Chi-square from the base model to the new model as each ingredient was added indicated how large a contribution the new variable made key contributions to post-treatment drug use.

Results: The results of this analysis show that the largest variance accounted for, by a large margin, was pre-treatment drug use (65.8%). Of the service ingredients analyzed, matched services (6%) and client-provider relationship (6%) were the most important ingredients.

Conclusions: The results support existing literature where targeted, client-center services and well as relational processes are emerging as significant mechanisms of change compared to intervention techniques or mechanisms (in this case substance counseling, 12-step programming and pharmacotherapy).