Measuring Organizational Health in Canadian Child Welfare Agencies Serving Indigenous Communities

Background and Purpose: Child welfare (CW) agencies must continually strive to maintain organizational health to provide the best care for their clients while meeting practice and policy standards. Organizational functioning is a multidimensional construct that impacts approach to practice, worker wellbeing, and client outcomes, highlighting the need for a validated assessment. However, some commonly known measures may present difficulties for CW agencies, as they often do not fully cover important dimensions of organizational functioning or are proprietary and costly. The Comprehensive Organizational Health Assessment (COHA) is a battery of measures developed to assess organizational health in CW settings, and has been tested in numerous settings including with two tribal CW programs in the US. However, use of the COHA has not been widely reported in Canada, and its use with Canadian Indigenous child welfare agencies is largely unknown. Recently, three CW agencies serving Indigenous communities in British Columbia sought to better understand their organizational environment and to test the use of the instrument in their settings. This first report examined the reliability and validity of the COHA in collaboration with these communities.

Method: Initially, the COHA was refined based on an extensive literature review and discussions with agency directors. The modified COHA was completed online by workers from the three agencies, as part of a mixed-method project examining relationships between organizational climate, approach to practice, and case outcomes. Internal consistency of the scales was tested using both Cronbach’s, to compare to previous literature, and ordinal alphas (OA) using polychoric correlations to account for ordinal items. Exploratory factor analyses (EFA) were conducted to examine the underlying structure of the scales, again using Pearson and polychoric correlation matrices. Finally, we examined interscale correlations between the COHA subscale scores and convergent validity between the COHA learning subscale score and the Dimensions of Learning Organizations Questionnaire (DLOQ).

Results: The sample (N=57) had an average of 8 years (SD=7.9) experience in CW and 42% identified as Indigenous. Across the 16 scales, Cronbach’s and ordinal alpha showed high internal consistency (0.736-0.973 and 0.721-0.975, respectively), with higher OA for all scales except for one. The Kaiser-Meyer-Olkin sampling for adequacy ranged from 0.439-0.936, and the Bartlett’s tests of sphericity were all statistically significant. Factor structures differed for 4 scales using Pearson as compared to polychoric correlations. Finally, there were moderate-to-strong correlations in the appropriate direction between COHA subscale scores and with the DLOQ, suggesting strong interscale validity and convergent validity on organizational learning dimensions.

Conclusions and Implications: The modified COHA had strong internal consistency and convergent validity on its learning organization dimensions. This assessment may be used to better understand the salient factors impacting organizational health for these three CW agencies serving Indigenous communities. This study highlights the use of polychoric correlations and OA when examining reliability and validity of ordinal-level items. Findings provide important information regarding research and practical implications for scale implementation. Identifying organizational strengths and limitations is essential for agencies to balance the needs of the communities and clients they serve with provincial guidelines.