Friday, 14 January 2005 - 2:00 PM

This presentation is part of: Adolescent Substance Abuse Prevention

Evaluation of Substance Abuse Prevention Programs: Increasing the Validity of Self-Report Data

Purpose: Program evaluation in the field of substance abuse prevention must be based on solid measurement and classification. Currently, self-report data are the mainstay of much social research. This is especially true for substance abuse prevention. The accuracy of self-reports in research is a complex issue. The more emphasis that researchers place on this method of data collection, the more it is important to ensure that the data obtained are valid. One suggested approach to increase the confidence of valid responses is to incorporate a “lie item,” such as a fictitious drug in substance use surveys. The present study sought to test this method to determine whether the inclusion of a “lie item” increases the validity of self-report data, and to determine if participants who report using a fictitious drug constitute a subpopulation that should be assessed differently.

Methods: This study was part of a statewide evaluation of substance abuse prevention programs. The sample comprised 894 youth in 6th-12th grade who completed pretest/posttest outcome instruments designed to measure 30-day use of substances. A fictitious drug identified as an enhancer (derbisol) was included. For each substance, youth rated their frequency of use on a 5-point scale from 1=daily (used about every day or at least 5 times a week in the last 30 days) to 5=never (never used in my lifetime). Participants who rated their derbisol use as 1-4 were placed in the Derbisol Group, and those who rated their derbisol use as 5 were included in the Non-Derbisol Group. T-tests were conducted to examine whether differences existed among the groups.

Results: Of the sample, 163 (18.2%) youth answered positively to using derbisol. For all drug types, there were significant differences between the two groups at both pretest and posttest. For every drug type, the Derbisol Group had significantly higher usage rates than the Non-Derbisol Group. When the 163 youth from the Derbisol Group were combined with the 731 youth from the Non-Derbisol Group, the only significant result was in the use of cocaine (t(893) = -2.52, p<.05), and the direction of change revealed increased usage. When data were analyzed for the Derbisol Group alone, significant change occurred only in the use of cocaine, with an increase in reported use (t(162) = -3.27, p<.01). However, when the Non-Derbisol Group was assessed by itself, marked differences were noted in the findings. Youth reported significant decreased use for smoking cigarettes (t(730) = 2.20, p<.05), alcohol (t(730) = 2.58, p<.05), and marijuana (t(730) = 2.54, p<.05). No significant differences were noted for the use of chewing tobacco or cocaine.

Implications: The results from this study suggest that those reporting use of the fictitious drug must be viewed as a unique and different subpopulation. Continuous efforts are needed by researchers to explore the patterns and trends of those reporting the use of a “lie item,” or otherwise providing misleading or invalid data. This type of assessment would enable researchers to more confidently infer which substance abuse prevention programs are working and under what circumstances.

See more of Adolescent Substance Abuse Prevention
See more of Oral and Poster

See more of Celebrating a Decade of SSWR (January 13 - 16, 2005)