Methods: CRI can be viewed as a process component in the CareerStart logic model, and its level in any classroom as intervention dosage. CRI is presently conceptualized as a teacher variable, but measured by student reports of how often teachers used examples from jobs and careers during instruction. Unlike its causal antecedent CareerStart, CRI was not assigned randomly, so a number of observed and unobserved factors influenced the dosage. To develop a causal estimate of CRI on achievement we began with bivariates of CRI and student, teacher and school characteristics. These establish the conditions supporting higher and lower CRI levels and identify a number of factors that may confound causal effects. Subsequently, we estimated the effect of CRI on achievement using three level models of students, teachers and schools with a large number of covariates at each level. We confirmed these findings using models employing an econometric technique known as the local average treatment effect (LATE).
Results: Bivariate relationships indicated the potential for a promotive effect of CRI on math achievement; alternatively, for reading, children reporting higher levels of CRI averaged lower EOG scores than children reporting lower levels. The multilevel models confirm that CRI predicts higher math achievement after controlling for: previous student performance; demographics; teacher effectiveness and other characteristics; school proportion minority and SES; and average school achievement. This finding is robust to different forms of the CRI measure: five-item score, a binary (high/low) CRI, and a teacher average. The LATE findings, which rely on the potential mediating relationship between CareerStart and CRI with respect to achievement, lend support to the inference that this is a causal effect.
Conclusions and Implications: Another paper within this symposium examines the nature and strength of the mediating relationship between CareerStart and CRI. These findings show that higher CRI promotes math performance and that relevance may be a cost effective strategy for education policymakers to consider. These findings also provide potential insight into why CRI does not seem to affect reading performance; if the lowest performing students consistently report high CRI we cannot expect to see an effect so long as CRI continues to be measured strictly by student perceptions. A further implication is therefore that multi-reporter and more valid measures of CRI must be developed.
