Innovators and Early Adopters in the Implementation of a Child Welfare Practice Innovation: Results of a Mixed Methods Study

Background and Importance

In 2014, the Virginia Department of Social Services (VDSS) developed a Learning Collaborative with child welfare agencies to identify behavioral indicators of enactment of their child welfare practice model, culminating in the development of 11 practice profiles. The primary method chosen for developing competency in the practice profiles was to coach in supervision and coaching training was provided to supervisors and senior staff in 2016 to approximately 50% of the state agencies in four cohorts. Research was conducted to understand the implementation process and outcomes of this effort, prior to engaging in a full state roll out of the practice. Informed by implementation science and diffusion of innovation theories and models, this study examines the differences between the Learning Collaborative agencies that were in the first training cohort (“Innovators”), and the other agencies that participated in Cohorts 1 and 2 (“Early Adopters”).

Methods

This study utilized an iterative mixed methods, longitudinal, non-experimental design. At project baseline, web-based surveys were distributed to child welfare staff at 18 randomly selected agencies representing various regions of the state, sizes of agencies, and training cohorts. Following the baseline survey, focus groups were conducted at participating agencies, which further informed the development of a second web-based survey that was conducted at 18 months post-training of agency staff. A second round of focus groups was also conducted immediately following the 18 month post survey. A final web-based survey was conducted at 18 months from the project baseline to serve as a post-assessment to the project baseline.

Results

Innovators had significantly higher levels of overall organizational readiness for implementation, as well as subscales measuring resources and capacity and community partnerships at baseline. Innovators also demonstrated higher levels of overall implementation climate, as well as for subscales measuring focus on, support for and selection for evidence based practice and selection for openness. These differences were no longer significant at 18 months post-training, however, 49% of Innovators reported use of coaching as opposed to 25% of Early Adopters and Innovators reported higher levels of agency consistency in implementation than Early Adopters.

Qualitative data from focus groups suggested that leadership at Innovator agencies felt a sense of ownership of the practice profiles and promoted them using different language than Early Adopter agencies. Innovator agencies had similar experiences to each other at 18 months, while Early Adopter agencies had more variation in implementation.

Conclusions and Implications

Data suggests that many of the differences that might have existed between agencies at the start of implementation faded overtime, as implementation efforts promoted readiness characteristics. However, a sense of ownership and personal identification with the practice – conveyed through participation in the Learning Collaborative - appeared to be an important factor and not replicated through training. These and other findings that will be shared are important for understanding the strengths and limitations of a “ground up” design approach when “going to scale.” Findings also have implications for key processes that support implementation of practice changes in child welfare.