Predicting Impact of Cross-Sector Training to Support Access to Behavioral Health Services

Background/Purpose: Access to effective youth behavioral health services continues to be a challenge. Outcome management tools support collaborative assessment and planning across human service sectors (child welfare, behavioral health, juvenile justice). Although the tools are integrated into policy, meaningful use of the tools (to engage youth and families, identify behavioral health and related functional problems, plan individualized interventions, and monitor progress) varies. 

Implementation strategies for the Child and Adolescent Needs and Strength Assessment (CANS) include online training and certification of reliability, in-person training of mid-level managers and supervisors as implementation coaches, booster’ workshops, and technical assistance. Little literature was found regarding cross-sector training for statewide implementation of a practice model in multiple organizations.

Research questions: What factors were associated with the transfer of knowledge from training to the meaningful use of the CANS in practice?

Methods: Measure. Since 2013, the Impact of Training and Technical Assistance (IOTTA) survey was completed at the end of each workshop. IOTTA questions include perceived mastery (pre- and post- workshop competence), training components (trainer credibility and organization, importance of goals, and interest), and the impact (improving reliability and utility, sharing information and modify use of tools in practice. Mastery, training, and basic impact items were rated on 11-point scales. Sharing and Use of information were rated on four-point scales. 

Data and samples. For October 2014 to October 2015 workshops, 1,113 participants from 73 organizations were classified into five types: 618 behavioral health, 320 child welfare supervisors, 44 child residential treatment clinicians, 53 other social service providers; and 78 unidentified. Most participants were Caucasian (85.3%) women (77%). About 10% were African American, 0.8% Native American, 0.8% Asian, 2.4% Hispanic, and 0.3% Other. 

Analysis. A multivariate linear regression analysis explored workshop and participant factors that may predict the overall Level of Impact of training on practice. Using backward entry, IOTTA items, agency type, participant characteristics (role, gender, race, ethnicity, language) and type of workshop created the initial model.

Results: A significant regression equation was found (F (6,945) = 501.864, p < .000), R² = 0.802).  For perceived Level of Impact, significant predictors included Existing Mastery (β = -.040, p = .011), Importance of Training Goals (β = .125 p = .000), Training Interest (β = .298, p = .000), Reliability and Utility Impact (β = .540, p =.000) Share Information (β = .091, p = .000) and Other Direct Service Role (β = -.047, p = .002). 

Interest in Training means differed significantly across types of agencies, χ2 (40) = 114.967, p < .000. Mean interest ratings follow (child welfare, 5.80; child residential, 7.44; mental health and addiction, 7.11; other social services, 7.44, and unknown, 6.34). 

Conclusions and Implications: Conceptually matching the practice model to agency functions is necessary in for effective implementation. Implications of the study suggest modifying the training format to increase interest through focused application of concepts in small agency specific groups followed by cross-system discussion to support collaboration. Social work is well positioned to conduct and apply translational research.