Four Guiding Principles for Quantitative Research Using Advanced Statistical Models

Background and Purpose

 Social worker researchers are increasingly applying advanced statistical models in their quantitative research. Due to the complexity and highly technical nature of the numerical approach embedded in a maximum likelihood (ML) method, researchers often face challenges to conduct the statistical analysis effectively, efficiently, and accurately. Taking the application of structural equation modeling (SEM) as an example, this workshop discusses four guiding principles for advanced statistical analysis, particularly analysis using the ML estimator.

Contents & Examples

 1.  A sound quantitative analysis should be guided by a theoretical model, and researchers should have rationale for all subjective decisions made throughout the data analysis. For instance, when running SEM, researchers need to impose constraints to make a underidentified model identified, need to free parameters such as non-correlational measurement errors to improve the goodness-of-fit of a model, in all these instances, researchers should always consider the plausibility of subjective decisions in a real world setting.

2. When facing multiple choices in modeling, other things being equal, researchers should always choose one that is most parsimonious. When formulating a research question, it is important to keep the research question simple. When the same research question can be answered by several models with different levels of complexity, it is important to choose the simpler model. The workshop uses an example of choosing between recursive and nonrecursive methods to show the importance of adherence to statistical parsimony.

3. Always run sensitivity analysis to check violations of assumptions embedded in a model. In quantitative research, a sensitivity analysis is the study of how the variation (uncertainty) in the output of a mathematical model can be apportioned, qualitatively or quantitatively, to different sources of variation in the input of a model. Using an example of scale development, the workshop illustrates how to run sensitivity analysis to address the problem of violating the multivariate normality assumption.

4. Always seek alternative explanations to a final model and run equivalent or competing models.  Closely related to the sensitivity analysis is the practice of running equivalent and competing models. This is often practiced after a final model is selected. In SEM, equivalent models yield the same predicted covariances but with a different configuration of paths among the same observed variables. In contrast, competing models may conceptualize the same set of variables based on a different theory, but yield different model fit indexes than the final model. No matter which model being tried, equivalent or competing, researchers should seek alternative explanations to make the final substantive conclusions rigorous, sound, reliable, and valid.

Significance

Through a nontechnical and intuitive explanation of SEM and ML, participants will learn the importance of having a substantively meaningful while statistically parsimonious model, caveats for conducting likelihood ratio tests, and strategies to avoid common pitfalls that help improve the quality of research findings.