Introduction: The combined problems of substance use and homelessness have been associated with increased emergency service usage, morbidity and mortality. Historically, high attrition rates and less rigorous methodology have hampered efforts to define what treatments are most effective. Over the past ten years, a number of interventions have been implemented and tested for this difficult to treat population. This is the first systematic review to evaluate the effectiveness of interventions for homeless persons with substance abuse problems that takes into account each study's methodological rigor.

Methods: This review identified 14 intervention studies published 1995-2004 based on a pre-determined set of literature search criteria. Studies were rated for methodological rigor, outcome significance, and treatment group comparisons. Evidence scores were calculated for each study, observation, and outcome. Types of interventions were grouped by treatment model (Intensive Case Management, Day Treatment, Day Treatment plus Residential, Low Demand Residential, and High Demand Residential). Studies were also grouped according to the percentage of dual diagnosis clients in the study population. A relational database, specifically designed for this review, was used as a tool for comparing and aggregating the evidence for each intervention outcome.

Results: Four of the fourteen studies reported significant improvement in substance use outcomes. Day Treatment and Day Treatment plus Low Demand Residential had the best evidence for positive outcomes in substance use. The Day Treatment model was effective for both dual diagnosis and substance abuse only clients. The High Demand Residential treatment model was the most effective intervention in improving housing outcomes. Significant improvements in housing outcomes were found only in studies that included a majority of dual diagnosis clients. Intensive Case Management, Day Treatment, and Low Demand Residential treatments were effective for this subset of the population. In terms of methodology, most studies used randomized design, made some account for attrition, and used rigorous analysis. The weakest methodological areas are in quality control, independent/blind follow-ups, and measurement rigor.

Discussion: Several conclusions can be drawn from this review. The Day Treatment models show promise in improving both substance use and housing outcomes for homeless persons. Surprisingly, there is no cumulative evidence for any of the other approaches in reducing substance use. The common use of less rigorous measures of substance use (i.e. single-item self reports) and high rates of attrition are problematic and when combined increase the likelihood of type II errors in evaluations of effectiveness. This may account for the fact that only 4 out of 14 studies found any significant improvement in substance use. Use of process evaluation techniques to improve treatment fidelity, measure contextual factors, and increase retention rates can enhance evaluations of treatment effectiveness.