Session: Supporting Change in Child Welfare: Evaluating the Children’s Bureau’s Training and Technical Assistance (T/TA) (Society for Social Work and Research 20th Annual Conference - Grand Challenges for Social Work: Setting a Research Agenda for the Future)

The federal government, states, foundations and others funders devote tens of millions of dollars in T/TA each year to support and strengthen health and human service systems, including child welfare agencies. There is limited research and evaluation studies about these “capacity building” efforts and whether they have been successful. This symposium presents the methodology and findings from a multi-year, mixed-method evaluation of T/TA  services provided by 15 federally-supported T/TA providers, the Children’s Bureau’s Child Welfare Implementation Centers (ICs) and National Resource Centers (NRCs), half of which were operated through University Schools of Social Work.  Centers provided T/TA to State and Tribal child welfare systems to help them achieve sustainable, systemic changes in order to improve child and family outcomes. NRCs delivered T/TA around child welfare topical areas and regional ICs provided T/TA through Implementation Projects that lasted at least 2 years and targeted specific change initiatives. Evaluation activities were influenced by implementation science and used multiple strategies to capture quantitative and qualitative data to examine the centers’ service provision, functioning, and effectiveness. The symposium papers highlight key process and outcome-related findings utilizing integrated data from multiple data sources and instruments. The four papers are complementary in that each builds upon the context established by the prior presentation. The first paper provides an overview of the Centers and provides an in-depth description of the types and characteristics of T/TA provided to child welfare jurisdictions using data from OneNet, a T/TA tracking database developed for the initiative. The second paper details the design of the cross-site evaluation of the Centers, the measures used to evaluate this comprehensive T/TA structure, and presents key evaluation findings, including sustained capacity building and systems changes reported by child welfare jurisdictions and contributions of T/TA to these changes. The third paper describes the development and findings from two cross-IC measures: an Implementation Process Rating Measure (IPM) to assess the salience and degree of installation of various drivers by implementation stage, and an Implementation Capacity Analysis (ICA) measure to assess jurisdictions’ perceived improvements in key capacity building domains. The fourth paper highlights the work of one IC and compares and contrasts jurisdictions’ and providers’ perceptions of T/TA, which provides insights into the usefulness of TA. The symposium explores the complexity and practical challenges of attempting to rigorously evaluate technical assistance, limitations of the evaluation, and lessons learned in hindsight.

The proposed symposium is of importance and significance to the child welfare research field, particularly those studying T/TA. The evaluation incorporated a unique method for operationalizing, characterizing, and measuring T/TA and its outcomes. Gaining greater understanding of the differences in perceptions between providers and recipients of TA offers valuable insights into how providers may improve TA effectiveness. In addition, aggregate data on T/TA needs, the organizational and systems changes occurring within child welfare systems, and the barriers and facilitators to change provide a snapshot of the current status of the field which can be informative in setting a future social work research agenda.