Evaluating the Rural Adaptation Project: Using an Interrupted Time-Series Design to Examine the Effects of Comprehensive Youth-Violence Prevention Programming in an Impoverished Rural County

Background and Purpose

The Rural Adaptation Project (RAP) is a 5-year panel study of over 6,000 youth in two rural counties in North Carolina. The aim of the RAP study is to reduce rates of youth violence by implementing three interventions: Positive Action (PA), a classroom-based violence-prevention curriculum; Teen Court (TC), a juvenile-justice diversion program for first-time offenders; and Parenting Wisely (PW), a computer-based parent-training program. The comparison county was chosen for its similarity to the intervention county in demographic and youth-violence indicators. In addition to conducting separate program evaluations for each of the aforementioned interventions, the study compared county-level indicators in the intervention county with indicators in North Carolina’s 99 remaining counties.

Methods

Using an interrupted time-series design, this study examines whether the direction of change in county-level indicators (i.e., delinquent complaints and undisciplined and delinquent complaints against juveniles) is consistent with the treatment effect of the RAP interventions. Specifically, we used annual data from 2004 to 2012 and a growth-curve model with random effects (i.e., hierarchical linear modeling, or HLM) to assess the impact of the intervention by evaluating the change of time-series data. It was hypothesized that (a) there will be a pattern of downward trends on county-level youth-violence indicators throughout the intervention period and into the follow-up year, relative to the indicators from pre-intervention baseline years; and (b) posttest scores on county-level youth-violence indicators will be significantly different from scores expected in light of the relationship between pretest and posttest scores in the state’s other counties.

Results

Overall, the examination of the general trends on undisciplined and delinquent complaints against juveniles confirms the first hypothesis: There is a pattern of downward trends on complaints against juveniles throughout the intervention period, relative to complaints in the pre-intervention baseline years. Over the study period, all three types of counties (the intervention county, urban counties, and rural counties other than the intervention county) showed decreases in the number of undisciplined and delinquent complaints against juveniles; however, the decline in the intervention county between 2010 and 2011 is steeper than declines in the urban counties and the remaining rural counties. Results indicate that there was an upward trend in delinquent complaints prior to 2010, and the results hold if the analysis controls for unemployment rate and child poverty in the intervention county only. This finding underscores the difficulty of addressing youth violence in this context. Between 2010 and 2011, however, delinquent complaints decreased and this decrease was steeper than that in North Carolina’s remaining counties.

Conclusion and Implications

Results provide preliminary support for the position that the RAP is effective at the county level. As social work researchers increasingly conduct county-level interventions, robust statistical methods offer a fruitful way to closely monitor changes in important community-level indicators such as the county-level youth-violence indicators in this study. Employing such methods enables researchers to gain robust evidence on intervention effectiveness while controlling efficiently for selection bias in a quasi-experimental study.