Organizational Supports and Evidence Use Among Private Child Welfare Agency Managers

Background

Private child welfare agencies (PCWAs) face substantial pressure to use evidence to spur innovation and improve client outcomes. Prior research in public health and health care suggests that organizational supports such as data systems or tools (technical infrastructure), staff (knowledge management infrastructure), policies and practices supporting evidence use (strategic alignment), and ties with external partners can facilitate staff evidence use. However, little is known about PCWAs’ strategic investments in such supports or their association with managerial evidence use. In this study, we used cluster analysis to describe and better understand PCWA investment in different organizational supports for evidence use. We also examined cluster-based differences in managerial evidence use.

Methods

Data are drawn from a 6-state survey of PCWAs (n=229; response rate=52%). Respondents were senior administrators and other key managers, who reported on agency supports for evidence use as well as their own evidence use. Hierarchical cluster analysis was used to categorize agencies into mutually-exclusive clusters, based on similarity in four types of organizational supports for evidence use (technical infrastructure, knowledge management infrastructure, strategic alignment, ties with external partners). Visual inspection of clusters and their distance measures as well as pseudo-F and pseudo-t² statistics were used to select well-defined clusters. Predictive discriminant analysis was used to validate the cluster solution. Multinomial logit models were used to identify cluster-based differences in three types of managerial evidence use (conceptual, persuasive, direct/process use). We controlled for contextual and organizational factors that influence evidence use such as institutional pressures, agency size, ownership, service diversity, membership in a larger network, and accreditation.

Results

Cluster analysis identified a five-cluster solution. Cluster 1 included PCWAs with above-average technical infrastructure but below-average use of other supports. These agencies typically offered highly diverse services. Cluster 2 included PCWAs with above-average technical infrastructure and strategic alignment but limited knowledge management infrastructure or ties with external partners. These agencies were large, with highly diversified services. Cluster 3 included smaller PCWAs with below-average investment in all organizational supports. Cluster 4 included larger PCWAs with above-average technical infrastructure and ties with external partners but below-average knowledge management infrastructure. Cluster 5 was comprised of PCWAs reporting above-average use of all organizational supports for evidence use. These agencies were smaller and offered less diverse services. Multinomial regression results suggest that relative to managers in Cluster 3 agencies, managers in Cluster 2 agencies reported higher conceptual and process use and managers in Cluster 4 agencies reported higher conceptual use. Managers in Cluster 5 reported high institutional pressures for evidence use, but did not differ from agencies in Cluster 3 in conceptual, persuasive, or direct use of evidence.

Conclusions/Implications

Study findings suggest that PCWAs vary considerably in their organizational supports for evidence use. Technical infrastructure coupled with either strategic alignment or external partnerships were associated with higher levels of managerial evidence use. In some smaller agencies, institutional pressures spurred agency investment in organizational supports for evidence use but did not impact actual use of evidence by managers. Implications for policy and agency practice will be discussed.