Developing Measurements for Analysis with Diverse Populations Using Item Response Theory and Structural Equation Modeling

Measuring health related concepts accurately is of critical importance for researchers and practitioners who deal with culturally diverse populations. Social workers in particular use measurements to assess health related outcomes in order to identify risk and protective factors for vulnerable, oftentimes historically disadvantaged populations. Meaningful, appropriate, and practical research instruments are not always readily available, or they may be misleading or biased. Although technological advances have enabled greater ease of data collection through computers and mobile devices, social work researchers must have both reliable and valid research instruments and measurements to ensure the validity of outcomes. Structural Equation Modeling (SEM) approaches such as Multi-sample Confirmatory Factor Analysis (CFA), Parallel Equivalence, and Tau Equivalence Tests can assess how well psychometric instruments capture latent psychosocial constructs across different populations. Item Response Theory (IRT) approaches can assess group differences in item and scale functioning while evaluating the person fit to improve existing measures for diverse populations. Additionally, IRT in conjunction with Computer Adaptive Testing (CAT) can potentially be used to calibrate answers to questions, by adapting the content of assessments to create better follow-up questions for respondents from diverse backgrounds. The combination of these approaches can lead to better comparisons across different cultural groups, which can help resolve fundamental issues of validity in cross-cultural social work research and evaluation.

This workshop aims to provide an overview of issues and techniques of cross-cultural measurement analysis using SEM and IRT approaches. Participants will learn step-by-step approaches of how to develop and assess cross-cultural measurements, which can be used to sidestep issues that can prevent the collection of reliable and valid data.  Participants will learn the different procedural steps in the testing of measurement equivalence hypotheses using SEM, such as testing the equivalence of covariance matrices of observed indicators, and parallel and Tau-equivalence tests of reliability of items across groups using Lisrel. The workshop will follow with an overview of IRT approaches, and participants will get hands-on experience in the use of basic features in IRTPRO®. Participants will develop a broad understanding in the central concepts of IRT, learn to interpret data generated from IRT analysis to inform tool development and refinement, and gain the skills to run basic IRT analysis. 

Topics    

Participants will learn:

1. The formulation of the research aims to the assessments of cross-cultural measurement properties.
2. The processes and issues of adopting or adapting existing research instruments for cross-cultural research.
3. The foundation of measurement theories and the process of instrument development from the definitions of abstract concepts, the construction of observed indicators, and assessment of the validity and reliability of the new instruments.  
4. The use of internal consistency analysis in assessing cross-cultural measurement comparability.
5. The use of multi-sample confirmatory factor analysis to evaluate cross-cultural measurement invariance of research instruments.
6. The application of Parallel and Tau-Equivalence Testing to evaluate true composite scores for multiple populations.
7. The use of Item Response Theory for assessment of group differences in item and scale functioning along with person fit.