Utilizing a Multi-Stage Validation Process with Exploratory Graph Analyses to Examine the Underlying Dimensionality of Factor Structures

Introduction: There is a consistent need to validate the factor structure of measures to ensure we accurately capture the dimensionality and nuances of a given construct. To aid in the advancement of measurement validation, this poster will demonstrate how the use of exploratory graph analyses (EGA) in combination with exploratory factor analysis (EFA) and confirmatory factor analyses (CFA) provides a detailed examination of the underlying dimensionality of a factor structure, adding more rigor to the measurement validation process.

Methods: Data from the 2021 National Survey of Children’s Health (NSCH) was utilized to ascertain the factor structure of 14 variables pertaining to young children's (aged 3-5, n = 12,000 ) social-emotional health. We employed EGA, EFA, and CFA. EGA provided network representations of the underlying structure, estimated using the Gaussian Graphical Model with EBICglasso regularization. Network stability was assessed with the Walktrap algorithm. After determining the structure, we investigated structural consistency using the bootEGA algorithm, generating 500 replicate datasets to assess the robustness of our findings. Subsequently, EFA started with unrotated extraction via Iterated Principal Factor Analysis and promax rotation to validate the theoretical structure. Lastly, a series of CFA models, using the robust maximum likelihood estimator (MLR), tested alignment with a predetermined conceptual structure. The evaluation included chi-square statistics, along with the comparative fit index (CFI), the Tucker-Lewis Index (TLI), and the Root Mean Square Error of Approximation (RMSEA). Decision-making criteria included network representation, item stability, factor loadings, and fit indices.

Results: EGA significantly enhanced the validation process. Initially, EGA revealed four dimensions within young children’s social-emotional health, categorizing 14 items into distinct domains, including 'self-regulation,' 'social competence,' 'emotional competence,' and 'behavior problems.' However, two items within the social competence domain displayed inadequate stability, falling below the stability threshold (< 0.70) required for further reliable factor identification. Recognizing this instability prompted an iterative process. EFA scrutinized item loadings, revealing issues within the four-factor model. Two items exhibited factor loadings below the recommended threshold of 0.32, leading to their deletion due to insufficient loading, refining the model. This iterative refinement resulted in a 12-item, four-factor model, confirmed by EGA to exhibit a more streamlined and interpretable set of factors. Finally, CFA demonstrated an optimal fit based on standard metrics of latent variable fit and parsimony. All items showed factor loadings above the cutoff of 0.40, indicating improved model precision.

Conclusion: The multi-stage validation process merged the strengths of traditional and contemporary factor analysis methods. Combining EGA with EFA and CFA allowed for integrating network theory benefits and traditional factor analysis, providing a nuanced perspective on item interrelationships. EGA, rooted in network theory, offers a fresh look at dimensionality, emphasizing complex item interactions overlooked by traditional methods alone. This approach ensured a systematic evaluation of dimensions and items, while maintaining theoretical coherence and statistical rigor throughout the process. This poster will depict how this novel approach may benefit social work research by enhancing the precision and accuracy of measurement validation, aiding in more valid and reliable assessments.