Abstract: Exploring District-Developed Report Card Indicators for Assessing Progress Toward Student Social Emotional Learning Goals (Society for Social Work and Research 22nd Annual Conference - Achieving Equal Opportunity, Equity, and Justice)

Schedule:

Friday, January 12, 2018: 8:00 AM

Independence BR A (ML 4) (Marriott Marquis Washington DC)

* noted as presenting author

Emily J. Campbell, MA, Doctoral Candidate, University of California, Berkeley, Berkeley, CA

B. K. Elizabeth Kim, PhD, Assistant Professor, University of Southern California, Los Angeles, CA

Juyeon Lee, MSW, Doctoral student, University of California, Berkeley, Albany, CA

Valerie Shapiro, PhD, Assistant Professor, University of California, Berkeley, Berkeley, CA

Background and Purpose: In order to scale up SEL programs, schools must develop an infrastructure that can support and evaluate SEL implementation (Elias et al., 2015). Some argue that integrating social-emotional indicators into report cards is the most pragmatic way to ensure social-emotional skills are assessed and nurtured like academic skills are (Elias, Ferrito, & Moceri, 2016). Districts are encouraged to create their own indicators that align with programming and learning goals, but these are often untested against basic psychometric standards. This paper seeks to determine how locally-developed report card indicators compare to a standardized tool for measuring social-emotional competence, such that high-quality information is used as the foundation for decision-making.

Methods: Data come from school records at a district serving 12,000 students (56% Latinx). Six elementary schools assessed students’ social emotional competence using two strategies. Report cards included 10 items (e.g., displays respectful behavior) that ask the teacher to rate the student’s proficiency level (1=unsatisfactory, 2=needs improvement , 3=satisfactory, 4=excellent) at the end of the first trimester, averaged into a Social-Emotional Report Card Composite (RCC). The DESSA-Mini included eight items (e.g., contribute to group efforts) that ask the teacher the frequency (0=never, 1=rarely, 2=occasionally, 3=frequently, 4=very frequently) of the student’s positive behaviors over the past four weeks, ultimately yielding a T-score (“Social Emotional Total”; SET) based on national norms. The DESSA-Mini has shown good reliability (Shapiro, Accomazzo, & Robitaille, 2017), discriminant validity (Shapiro & LeBuffe, 2006), sensitivity, specificity, and predictive validity (Shapiro et al., 2016; Chain et al., 2017).

Results: The RCC (𝛼=.93) was completed on 3,201 students (M=3.28; SD=.56). Girls were more proficient than boys (p<.001; d=.50). There were also significant differences between children eligible and ineligible for free lunch (p<.001; d=.16) and marginally significantly differences between White and non-White students (p<.059; d=.11). Thirty percent of the variance in RCC scores was explained at the teacher level. The SET (𝛼=.95) was completed on 3,181 students (M=50.99; SD=11.69). Girls were more proficient than boys (p<.001; d=.47). There were significant differences between children eligible and ineligible for free lunch (p<.001; d=.17), but not between White and non-White students (p=.79). Eighteen percent of the variance in SET scores was explained at the teacher level. The RCC and SET scores were significantly related (r=.57, p<.001).

Conclusions and Implications: As expected, report card indicators and standardized ratings of social-emotional competence were moderately correlated, indicating convergent validity, but also unique contributions. However, some advantages to using standardized measures such as the DESSA-Mini, rather than ad-hoc indicators on report cards, were discovered. The DESSA-Mini was more internally reliable than the report card indicators, with smaller group differences detected by gender and race, and more of the score attributable to the student rather than the rater. Implications of persistent differences by gender and poverty status will be discussed.