Determining Measurement Invariance in Social Work Research

Background and Purpose:  The use of psychosocial measures with different conceptual meanings across cultural groups may render treatment outcome analyses invalid in social work research. Determining measurement invariance of an instrument allows researchers to assess whether the construct of a measure is similarly comprehended and measured across participant groups (e.g., based on race, ethnicity, gender, age, etc.). Measurement nonequivalence is introduced when groups of participants experience or conceptualize a construct differently, or use distinctive criteria to describe the construct. Measurement nonequivalence across cultural groups is posited to occur due to (a) cultural differences in norms and relevance of the constructs being assessed; (b) language of assessment; or (c) potential differences in participants’ environments and opportunity structures to engage in certain behaviors or develop beliefs due to contextual differences, racism, or other forms of discrimination. Thus, participants from different cultural groups may respond to item content in dissimilar ways causing variable construct validity across groups.  This presentation will examine measurement invariance properties across racial groups for two commonly used instruments in social work and substance abuse treatment research: The Revised Helping Alliance Questionnaire (HAq-II) and The Short Inventory of Problems (SIP).

 Methods: Publically available, secondary data were obtained from a randomized clinical trial conducted in the National Institute on Drug Abuse Clinical Trials Network. The trial investigated the effect Motivational Enhancement Therapy (MET) on retention and substance use among adults seeking treatment at five community-based outpatient treatment programs. To assess invariance across racial groups, HAQ-II data were examined for 138 African American and 133 non-Latino White participants at their second MET session.  Baseline SIP data were examined across 195 African American participants and 194 non-Latino White participants.

 Results: The HAq-II demonstrated configural invariance and two forms of metric invariance, suggesting that conceptualizations of therapeutic alliance and overall levels of endorsement of therapeutic alliance are comparable across racial groups. The groups also indicated partial strict metric nonequivalence for the HAq-II.  The SIP demonstrated configural invariance and weak metric invariance, suggesting conceptualizations of adverse consequences of substance use are equivalent across racial groups. The SIP also indicated partial strong/scalar and strict metric invariance, suggesting a need for continued research of SIP items to ensure valid measurement and outcomes across racial groups.

 Conclusions and Implications: Many psychosocial measures used in social work research are susceptible to measurement nonequivalence because measurement nonequivalence is frequently overlooked by researchers. The continued use of measures with different conceptual meaning across racial and ethnic groups may render invalid analyses comparing such groups. Conclusions drawn from invalid findings can lead to ineffective treatments and policy initiatives. Yet, few studies have reported measurement equivalence properties of widely used outcome measures across salient groups.  This presentation will illustrate the rationale for, and basic analytic steps to conduct, measurement invariance analyses as well as interpret findings.