Bridging Disciplinary Boundaries (January 11 - 14, 2007)

Objective. The purpose of this study was to evaluate potential sources of bias in a longitudinal study of an adolescent drug treatment program in a rural state. We examined recruitment, retention and treatment integrity as potential sources of bias in our overall research.

Method. This study examined data collected between 2003 and 2005 from adolescents who presented for suspected substance abuse at an outpatient assessment and treatment center in eastern Iowa. To evaluate the impact of recruitment and retention, we focused on participant characteristics (e.g., gender, age, race, etc.), mental health diagnoses and substance use disorders for participants in two different treatment styles at 3 and 6 month follow-up points. To evaluate treatment integrity, we compared service units (e.g., individual counseling, family therapy, group treatment, etc.) between adolescent-only group treatment and strengths oriented family therapy.

Results. The two treatment groups were similar on all demographic characteristics at recruitment. Overall, those who declined to participate in our longitudinal study had lower severity of substance frequency, substance related problems and severity of emotional problems but only the difference for emotional problems was significant (with participants having more emotional problems). We continued to compare those retained with those missing at 3 and 6 months using baseline (intake) data. At 3-months, we retained 89% of the 172 eligible for evaluation, and found no differences in intake scores between those retained and those missing. At six months, we found that those who were retained (86% retained) had significantly less substance use problems and significantly lower substance use frequencies than those who were missing (again comparing intake data). In addition, those retained at 6-months were significantly less likely to be diagnosed with conduct disorder and traumatic stress disorder that those missing. Finally, we found that actual service units for each treatment approach were appropriate for the model being delivered.

Conclusion. Many of the potential sources for bias associated with longitudinal research were initially controlled through the research design. Although the recruitment rate was about 50%, our data suggest that participants were equivalent to decliners. Overall retention rates were at 89% and 86% for 3 and 6-month follow-up points which strengthen results from our evaluation of treatment effectiveness. Further, by comparing those retained with those missing at each follow-up point, we can then include factors with potential bias in data analysis models. Monitoring of treatment delivery assured adherence to the treatment model. Finally, as reported elsewhere, our 3 and 6-month data suggest significant reductions in substance use frequencies and substance abuse problems for participants in both treatment groups.

Implications for Policy & Practice. Longitudinal research on adolescent substance abuse has been hindered by bias related to recruitment and retention problems, and from treatment implementation issues. This study identified effective methods of minimizing those forms of bias, resulting in more accurate reporting of outcomes.