Methods. In 2005-2006, we collaborated with a North Carolina school district to develop and introduce an intervention called CareerStart to seven randomly assigned middle schools (with seven control schools), and tracked an initial cohort of 2,500 students for a period of three years. We hypothesized that CRI would improve school engagement (a three item scale) and valuing (a seven item scale) of students in the treatment schools relative to students in the control schools. We used a mixed methods design, including focus groups and student surveys of perceived levels of CRI as well as administrative data. We employed intent-to-treat (ITT) analyses of CareerStart and treatment on the treated (TOT) analysis of school mean CRI as a measure of the dosage. For the analyses of student perceptions of CRI we used ANCOVA and hierarchical linear modeling (HLM), for ITT we used HLM, and for TOT we used HLM with an econometric approach known as instrumental variables (Ebbes, Bockenholt & Wedel, 2004).
Results. CRI promotes significantly higher school student engagement and school valuing, particularly among minority and low-income students. While the ITT analyses failed to identify a significant effect for the treatment, this may be due to the under-powered design (14 schools) of the evaluation. Alternatively, the TOT analyses confirm the student-level perception results, indicating that higher levels of CRI yield greater valuing of education among 7th grade students. Qualitative data indicate that students find the focus on relevant education engaging, thus confirming the quantitative findings.
Conclusions and Implications. This study has broad policy and research implications. We hypothesize that student school engagement and valuing are mediators of end-of-grade performance, which we propose as the next step in the analysis. This universal, pedagogical approach was introduced by a social work and education team and has demonstrated that this collaboration can positively affect the psycho-social outcomes that are foundational to academic achievement and school retention. These early results have already led to six school districts and 14,500 students participating in the program.