How to Do Differential Item Functioning (DIF) Studies Using Item Response Theory

Numerous measures are used in social work research and practice.  It is critical that the validity of scores from these measures be invariant across the diverse populations who participate in research and for whom social workers provide services.  A lack of invariance would mean the scores on these measures are biased, must be interpreted differently for persons from the different populations, and may lead to erroneous inferences in practice and research.

A powerful methodology for identifying biased items is the “differential item functioning,” or DIF, study.   When items function differently for different populations, it is referred to as DIF.  DIF is a necessary but not sufficient condition for item bias.  The results of DIF studies can be followed by investigations into the sources of, and the elimination of, item bias.   

Item Response Theory (IRT) is an extension of classical measurement theory, and a fundamental IRT concept, the “Item Characteristic Curve” (ICC), can be used to define two forms of DIF: uniform and non-uniform.  IRT methods can also be used to detect DIF in dichotomously scored items.  An alternate approach using logistic regression can be used to identify DIF in both dichotomous and polytomously scored items. 

This workshop will provide participants with the knowledge necessary to use both IRT and logistic regression to conduct DIF studies.  First, the IRT concept of the ICC will be presented and used to define “uniform” and “non-uniform” DIF.  The use of IRT methods, using MULTILOG, for identifying items that exhibit DIF will be introduced.  Participants will be given a step-by-step procedure for using MULTILOG to fit two-parameter logistic ICCs to item scores and the use of two methods for identifying DIF: comparing ICC parameters, and computing the area between ICCs, for different populations.  This procedure will be modeled for participants by showing its use, from beginning to end, to analyze item scores from dichotomous items on the Geriatric Depression Scale (GDS). 

Participants will next be shown, in the same pedagogic manner, how to use binary logistic regression in SPSS to identify uniform and non-uniform DIF in scores from GDS items, and how to use the results of these analyses to plot ICCs for items exhibiting DIF.  Participants will also be shown how to use ordinal logistic regression to test for uniform and non-uniform DIF in the responses to polytomously scored items.  This approach will be demonstrated using scores from the depression subscale on Hudson’s MPSI. 

Participants will learn how to follow-up the identification of items exhibiting DIF with research to identify whether bias exists in the item, the sources of the bias, and what can be done to eliminate the bias.  Participants can expect to leave this workshop with a basic understanding of the conceptualization of DIF using ICCs; beginning skills in the use of IRT and logistic regression for conducting DIF studies; and the ability to use results from DIF analyses for developing measures for use in social work research and practice that are unbiased, and culturally and population sensitive.