IRT Analysis of the Devereux Student Strengths Assessment in a Special Need Sample

Purpose:

Social and emotional skills are critical to youth’s academic and social development (Cohen, Onunaku, Clothier, & Poppe, 2005; Jones, Bouffard, & Weissbourd, 2013).  With the adoption of social and emotional learning (SEL) standards alongside academic standards as critical outcomes of the educational process school social workers are increasing involved in developing and implementing psycho-social interventions that promote SEL (Dusenbury, Weissberg, Goren, & Domitrovich, 2014).  The ability to have measures that have psychometric properties that will allow accurate assessment of the outcomes of SEL interventions is critical.  The Devereux Student Strengths Assessment (DESSA) is especially suited to support evaluation of school social workers efforts to evaluate interventions targeting SEL. DESSA is a strength-based, norm referenced comprehensive measure of SEL.  Basic psychometric information (reliability and predictive validity) for youth in general education settings is available on the DESSA (Haggerty, Elgin, & Woolley, 2011; Naglieri, Goldstein, & LeBuffe, 2010).  But little is known about the psychometric properties of the measure with youth participating in special educational programming.  This study conducted an Item Response Theory (IRT) analysis of the DESSA in a sample of 436 middle school youth enrolled in a special educational program who all had an emotionally disturbed (ED) classification. 

Methods:

            DESSA is a norm referenced behavioral rating scale (Items=72) that assessed the youth social-emotional competencies (LeBuffe, 2008).  School personal, parents, after-school staff or any person who has knowledge of the youth’s behavior can complete the measure.  Using a 5 point rating scale which indicates the frequency of occurrence of a particular behavior.   The 72 items are divided into eight subscales each social and emotional competencies: Self-Awareness; Social Awareness; Self-Management; Goal-Directed Behavior;   Relationship skills; Personal Responsibility; Decision Making and Optimistic Thinking.

Factor analysis was used to examine the unidimensionality of the subscales. Two-parameter graded response (2PL) IRT models for were estimated for each subscale.  Mplus 7.3 was used to estimate all factor analysis and the IRT models with the weight least squares means and variance estimator.

Results:

            The IRT analysis provides evidence that items varied in the degree of discrimination and the test information curves for the items indicted acceptable levels of reliability (Alpha of .80 or higher) across the levels of the dimensions being measured.   The study provide evidence that the items on the DESSA and their associated subscale can discriminate well across levels of the SEL competencies being assessed.  

Implications:

School social worker seeking to engage in evidence informed practice need measures with psychometric properties tailed to where youth may be on the construct being measured.  This study provides that evidence for the use of the DESSA with sample of student classified as ED.  For those seeking to use item level information to target interventions the results of this study identified the most and least discriminating items on the measure.   Knowledge of the discriminating power of items based on the level at which a youth has acquired a competency being measures will contribute to improving the quality intervention outcome data being gathered by social worker.