Measuring School Engagement of Early Adolescents: Invariance Testing of the SSP School Satisfaction Scale

Background: School engagement is a powerful predictor of numerous academic, social, emotional, and health outcomes in adolescence and adulthood. Although many measures of engagement exist, few have been examined for measurement equivalence. The purpose of the current study is to examine the quality of data collected using an existing measure of engagement and determine whether the scale operates equivalently with early adolescents from four different racial/ethnic groups. Measurement invariance across groups permits greater confidence in the measure’s appropriateness for valid assessments, cross-group comparisons, and tests of substantive hypotheses.

Methods: As part of the North Carolina Rural Adaptation Project, the 7-item School Success Profile (SSP) School Satisfaction scale was administered with 2,063 students in grades 6-8 from 28 public schools in two rural, economically disadvantaged counties. The sample was racially/ethnically diverse: 33% Native American, 30% White, 27% African American, 10% Hispanic/Latino. A multiple group confirmatory factor analysis was conducted in Mplus 7 to test for measurement invariance. Weighted least squares means and variances (WLSMV) estimation was used because the data are both non-normal and ordinal. Clustering of students within schools was modeled. Invariance testing followed current recommended procedures for estimation with ordinal variables, including modeling of thresholds (Byrne, 2012; Muthén & Muthén, 1998-2012; Sass, 2011).

Results: Scale items displayed configural invariance, indicating that the basic factorial structure was the same across the four racial/ethnic groups. Results revealed noninvariance for one factor loading. The loading was substantially lower for Hispanic/Latino youth, suggesting that the idea captured by the indicator (i.e., enjoyment of going to school) contributes less to Hispanic/Latino’s latent engagement score than it does for students of other racial/ethnic groups. Although full scalar invariance was not supported (a small percentage of thresholds were not fully invariant across the four groups), the establishment of partial measurement invariance permitted cross-group comparisons of latent means. Engagement differed significantly across racial/ethnic groups. Caucasian and Hispanic/Latino students reported significantly higher levels of engagement than African American students. Native American students’ engagement levels did not differ significantly from any other racial/ethnic group. Further post-hoc analysis (Sass, 2011) indicated that the identified areas of non-invariance did not affect conclusions about latent mean differences.  

Conclusions/Implications: As a potential protective factor, school engagement is a target for many prevention/intervention efforts. Evaluating such programs and detecting real change in targeted constructs depends, in part, on using measures that perform well psychometrically. This study contributes to intervention and evaluation efforts by validating a scale to assess levels of engagement and to make cross-group comparisons among racially/ethnically diverse early adolescents. Study findings are consistent with SSP developers’ initial examination of the scale and with the small number of studies that examine measurement equivalence of other engagement scales. Many studies of engagement do not employ diverse samples; as such, this study adds to our understanding of engagement and potential differences across racial/ethnic groups. This study also contributes to the measurement literature by demonstrating the most current and rigorous methods for establishing measurement invariance with non-normal ordinal variables.