Conducting DIF and Dtf Analyses Using Irtpro

Background and purpose: Numerous Likert type measures are used in social work research and practice.  It is critical that the validity of scores from these measures be invariant across the diverse populations who participate in social work research and for whom social workers provide services.  A lack of validity invariance would indicate scores on these measures represent different constructs and need to be interpreted differently for persons from the different populations.  This lack of validity invariance, or differential validity, may lead to erroneous inferences.

A methodology for identifying differential validity of items is the differential item functioning, or DIF, study.  When items function differently for different populations, it is referred to as DIF. DIF can propagate to the total score on a scale producing differential test functioning, or DTF, meaning the total score on a scale represents different constructs for different populations.  Item Response Theory (IRT) is a powerful approach for conceptualizing and detecting DIF and DTF.  This workshop will provide participants with a basic knowledge of: (a) DIF and DTF through the lens of IRT concepts, (b) how IRT can be used to detect DIF and DTF in Likert type scales using the program IRTPRO, and (c) how to use a free online Excel program to estimate effect sizes for DIF and DTF. 

Methods: First, the basic IRT concepts of the item response function and the item characteristic curve will be introduced and used to define two forms of DIF, uniform and non-uniform.  The IRT concepts of the item and test information curves will be used to illustrate how DIF may propagate to the total scale score to produce DTF, thereby posing a serious threat to validity.  Recently developed effect sizes for DIF and DTF will be described, and participants will be shown how to use the Excel program to obtain estimates of these DIF and DTF effect sizes.  Participants will be walked step-by-step through a DIF analysis of 12 Likert items from the Depression Subscale on Hudson’s Multi-Problem Screening Inventory (MPSI) using IRTPRO.  Participants will also be walked step-by-step through use of the Excel program to obtain effect size estimates for DIF and DTF.  

Results: Participants will learn how to conduct, and interpret the results of, confirmatory DIF and DTF analyses.  Participants will also learn how to follow-up the identification of items exhibiting DIF with subsequent research to identify whether bias exists in the items showing DIF; to identify sources of the bias; and what can be done to eliminate the bias.  

Conclusions and implications: Participants can expect to leave this workshop with a basic understanding of foundational IRT concepts and the IRT conceptualizations of DIF and DTF; beginning skills in the use of IRT-PRO for conducting DIF and DTF studies; the use of the Excel program for estimating DIF and DTF effect sizes; and the ability to use the results from DIF and DTF analyses for developing measures for use in social work research and practice that are unbiased and culturally and population sensitive.