Society for Social Work and Research

Sixteenth Annual Conference Research That Makes A Difference: Advancing Practice and Shaping Public Policy
11-15 January 2012 I Grand Hyatt Washington I Washington, DC

16082 The Causal Role of Career Relevant Instruction In Promoting Middle School Academic Achievement

Friday, January 13, 2012: 8:00 AM
Arlington (Grand Hyatt Washington)
* noted as presenting author
Roderick A. Rose, MS, Research Associate, University of North Carolina at Chapel Hill, Chapel Hill, NC
Michael Woolley, DCSW, PhD, Associate Professor, University of Maryland at Baltimore, Baltimore, MD
Dennis K. Orthner, PhD, Professor, Associate Director for Policy Development and Analysis, Jordan Institute for Families, University of North Carolina at Chapel Hill, Chapel Hill, NC
Hinckley Jones-Sanpei, PhD, Consultant, University of North Carolina at Chapel Hill, Chapel Hill, NC
Background and Purpose: CareerStart is an evidence-based middle school instructional intervention designed to promote the use of relevance through career examples in lesson content. Motivation theory asserts making the material relevant to students' futures will improve engagement and academic achievement. A randomized control trial confirms that CareerStart promotes the use of career relevant instruction (CRI) among math teachers, and improved achievement among middle school students in end-of-grade (EOG) math tests. However, questions remain about whether CareerStart causes higher math achievement through CRI, and why CareerStart did not affect reading achievement. Understanding these mechanisms is critical to the ongoing development of CareerStart to better serve the needs of youth at-risk for school failure.

Methods: CRI can be viewed as a process component in the CareerStart logic model, and its level in any classroom as intervention dosage. CRI is presently conceptualized as a teacher variable, but measured by student reports of how often teachers used examples from jobs and careers during instruction. Unlike its causal antecedent CareerStart, CRI was not assigned randomly, so a number of observed and unobserved factors influenced the dosage. To develop a causal estimate of CRI on achievement we began with bivariates of CRI and student, teacher and school characteristics. These establish the conditions supporting higher and lower CRI levels and identify a number of factors that may confound causal effects. Subsequently, we estimated the effect of CRI on achievement using three level models of students, teachers and schools with a large number of covariates at each level. We confirmed these findings using models employing an econometric technique known as the local average treatment effect (LATE).

Results: Bivariate relationships indicated the potential for a promotive effect of CRI on math achievement; alternatively, for reading, children reporting higher levels of CRI averaged lower EOG scores than children reporting lower levels. The multilevel models confirm that CRI predicts higher math achievement after controlling for: previous student performance; demographics; teacher effectiveness and other characteristics; school proportion minority and SES; and average school achievement. This finding is robust to different forms of the CRI measure: five-item score, a binary (high/low) CRI, and a teacher average. The LATE findings, which rely on the potential mediating relationship between CareerStart and CRI with respect to achievement, lend support to the inference that this is a causal effect.

Conclusions and Implications: Another paper within this symposium examines the nature and strength of the mediating relationship between CareerStart and CRI. These findings show that higher CRI promotes math performance and that relevance may be a cost effective strategy for education policymakers to consider. These findings also provide potential insight into why CRI does not seem to affect reading performance; if the lowest performing students consistently report high CRI we cannot expect to see an effect so long as CRI continues to be measured strictly by student perceptions. A further implication is therefore that multi-reporter and more valid measures of CRI must be developed.

Previous Abstract | Next Abstract >>