The Use of Item Response Theory and IRT-PRO to Conduct Differential Item Functioning (DIF) Studies of Likert Type Items

Numerous measures are used in social work research and practice, and many use polytomous items.  It is critical that the validity of scores from these measures be invariant across the diverse populations who participate in research and for whom social workers provide services.  A lack of invariance would indicate scores on these measures are biased, must be interpreted differently for persons from the different populations, and may lead to erroneous inferences in both research and practice.

A methodology for identifying potentially biased items is the differential item functioning (DIF) study.  When items function differently for different populations, it is referred to as DIF; DIF is a necessary but not sufficient condition for item bias.  The results of DIF studies can be followed by investigations into sources of, and the elimination of, biased items.   

Item Response Theory (IRT) is a powerful approach to detecting DIF.  IRT methods can be used to detect DIF in Likert type and other forms of polytomous items.  This workshop will provide participants with basic knowledge of fundamental concepts from IRT and how IRT can be used to detect DIF in Likert type items.  Participants will also learn how to use the computer program IRT-PRO to conduct DIF analyses of Likert type and other polytomous items. 

First, the IRT concepts of the item response function and the item characteristic curve will be introduced and used to define two forms of DIF, uniform and non-uniform.  The IRT concept of the test information curve will also be introduced and used to illustrate how DIF may propagate to the total scale score, thereby posing a serious threat to validity. 

Participants will be given a step-by-step procedure for conducting a DIF analysis of Likert items.  This approach will be illustrated by first showing participants how to use ordinal logistic regression to do preliminary analyses to identify anchor items, items with no evidence of DIF.  Participants will then be shown how to use IRT-PRO to do a confirmatory DIF analysis by conducting a DIF analysis of item scores from the Depression Subscale on Hudson’s Multi-Problem Screening Inventory (MPSI).  Participants will learn how to use the identified anchor items when using Samejima’s graded response model in IRT-PRO to detect the presence of DIF in MPSI Depression subscale scores, and how to use the test information curve to detect possible threats to validity of total scores on the scale.

Participants will learn how to follow-up the identification of items exhibiting DIF with subsequent research to identify whether bias exists in the items showing DIF; and identify sources of the bias, and what can be done to eliminate the bias.  Participants can expect to leave this workshop with a basic understanding of basic IRT concepts and the IRT conceptualization of DIF; beginning skills in the use of logistic regression and IRT-PRO for conducting DIF studies; and the ability to use results from DIF analyses for developing measures for use in social work research and practice that are unbiased, and culturally and population sensitive.