Item Response Theory and Its Use In Differential Item Functioning (DIF) Studies

Item Response Theory (IRT) is a powerful extension of classical measurement theory. If the assumptions of IRT are met, IRT enables the researcher to develop and evaluate measurement tools for use by practitioners that cannot be developed within a classical framework. For example, IRT can be used to evaluate the extent to which scores on items on a measure must be differentially interpreted for persons from different groups, such as males and females or persons from different cultural groups. Studies of possible “differential item functioning” are referred to as DIF studies. Thus, IRT can be used to develop measurement tools that are critical for culturally, and population, sensitive assessment. This workshop will introduce participants to the fundamentals of IRT, to the use of the IRT program MULTILOG, and to the use of IRT to conduct DIF studies. First, basic concepts such as the item response function, the item response curve, the theta metric, the test characteristic curve, and item and test information functions will be discussed and illustrated. These concepts will be introduced and exemplified throughout the workshop by comparing and contrasting these IRT concepts with fundamental concepts from classical theory, such as reliability. The emphasis in this portion of the workshop will be to inculcate in participants a conceptual understanding of these fundamental IRT concepts; how they differ from classical theory tenets; and most importantly how participants can use these IRT concepts to understand (a) how items may function differently with persons from different populations, and (b) how such differential item functioning may be detected using IRT methods. Participants will also be introduced to the use of the program MULTILOG by way of an illustrative use of the program for conducting a DIF study of scores from items on a brief suicidal thinking screener. Participants will learn how to set up data for MULTILOG; how set up a MULTILOG analysis to obtain item parameter estimates; and how to use these estimates to obtain estimates of item and test information curves. These procedures will be illustrated for both a dichotomous two parameter logistic IRT model, and for scores from a Likert type scale using Samejima's graded response IRT model. Participants will then be taken step-by-step through a DIF study using a two parameter logistic IRT model with dichotomous data from a brief suicidal thinking measure. Participants will learn how to set up IRT analyses using MULTILOG to test for DIF in the so-called “difficulty” (or b) parameters; and for DIF in the so-called “item discrimination” (or a) parameters. Participants will also learn how to interpret the results from these analyses concerning the presence or absence of DIF. Participants in this workshop can expect to leave with a basic grasp of fundamental IRT concepts; an understanding of how to use MULTILOG for IRT analyses; and a basic understanding of how to use IRT via MULTILOG to conduct DIF analyses and use the results to develop more culturally and population sensitive measures for use in social work practice.