Application of Ordinal Response Models of Item Response Theory in Social Work Research

Background

Item Response Theory (IRT) is a psychometric technique commonly used in development of instruments, ability and perception assessment, and test scoring. Specifically, IRT aims to examine the properties of latent traits of populations based on categorical response data. IRT has been recognized as a valuable psychometric technique and is widely used in psychology, education, political science, and many other disciplines. However, in social work, there has been a minimal number of studies drawing on IRT to substantiate instruments or assess latent traits, abilities, or constructs. Moreover, beyond statistical inferences, the value of IRT in informing group differences in item and instrument functioning and refining existing measurement for diverse populations, especially those underserved and underrepresented, has been overlooked. Thus, there remains a great need for a clear, concise, and application-oriented overview of IRT models in social work research.

Objectives

Upon completion of the workshop, participants will be able to:

1. demonstrate understanding of commonly used ordinal response models of IRT (ORM) and evaluate their relevance to instruments used in social work research and practice;

2. perform ORM analyses and generate test and item characteristic curves using R;

3. detect differential item functioning (DIF) based on multiple group IRT model analysis;

4. interpret model diagnostic and fit statistics and parameter estimates; and

5. derive inferences and inform refinement of instruments.

Content

Based on the pedagogical techniques of learning-science-by-doing-science (Labouta et al., 2018) and problem-based learning (Savery, 2006), this workshop aims to address the following content:

1. introduction to IRT including its history, typology, assumptions, and application;

2. the significance of IRT in social work research within contexts of instrument development and assessment of latent constructs;

3. an evidence-based, delineated, and systematic set of guidelines to identify appropriate IRT models to accommodate the needs of social work researchers, faculty, students, and professionals;

4. application of IRT models in social work research and a problem example based on real data;

5. step-by-step demonstration of model fitting and plotting using R and interpretation of relevant statistics including DIF identification; and

6. drawing inferences and conclusions to inform refinement of instruments.

This workshop will encourage questions and comments throughout, and the pedagogical approach will exploit figures and illustrations to facilitate interpretation. All participants will receive a paper and/or electronic copy of a handout detailing the workshop content including slides, syntax, best practices, and a list of relevant resources.

Implications

Overall, this focus on IRT models can be invaluable to social work researchers, faculty, students, and professionals who wish to develop, appraise, or refine psychometric tools. Featuring real data analyzed with a commonly used statistical software application, this workshop simplifies esoteric statistical models making them more applicable to social work research. By becoming familiar with best practices in applying IRT models, attendees can gain the expertise necessary to use more modern psychometric models that advance research and further development of IRT modeling.