Session: Supporting Change in Child Welfare: Evaluating the Children’s Bureau’s Training and Technical Assistance (T/TA) (Society for Social Work and Research 20th Annual Conference - Grand Challenges for Social Work: Setting a Research Agenda for the Future)

146 Supporting Change in Child Welfare: Evaluating the Children’s Bureau’s Training and Technical Assistance (T/TA)

Friday, January 15, 2016: 5:15 PM-6:45 PM
Meeting Room Level-Meeting Room 15 (Renaissance Washington, DC Downtown Hotel)
Cluster: Child Welfare
Symposium Organizer:
James DeSantis, PhD, James Bell Associates
Brian Deakins, MSW, Children's Bureau
The federal government, states, foundations and others funders devote tens of millions of dollars in T/TA each year to support and strengthen health and human service systems, including child welfare agencies. There is limited research and evaluation studies about these “capacity building” efforts and whether they have been successful. This symposium presents the methodology and findings from a multi-year, mixed-method evaluation of T/TA  services provided by 15 federally-supported T/TA providers, the Children’s Bureau’s Child Welfare Implementation Centers (ICs) and National Resource Centers (NRCs), half of which were operated through University Schools of Social Work.  Centers provided T/TA to State and Tribal child welfare systems to help them achieve sustainable, systemic changes in order to improve child and family outcomes. NRCs delivered T/TA around child welfare topical areas and regional ICs provided T/TA through Implementation Projects that lasted at least 2 years and targeted specific change initiatives. Evaluation activities were influenced by implementation science and used multiple strategies to capture quantitative and qualitative data to examine the centers’ service provision, functioning, and effectiveness. The symposium papers highlight key process and outcome-related findings utilizing integrated data from multiple data sources and instruments. The four papers are complementary in that each builds upon the context established by the prior presentation. The first paper provides an overview of the Centers and provides an in-depth description of the types and characteristics of T/TA provided to child welfare jurisdictions using data from OneNet, a T/TA tracking database developed for the initiative. The second paper details the design of the cross-site evaluation of the Centers, the measures used to evaluate this comprehensive T/TA structure, and presents key evaluation findings, including sustained capacity building and systems changes reported by child welfare jurisdictions and contributions of T/TA to these changes. The third paper describes the development and findings from two cross-IC measures: an Implementation Process Rating Measure (IPM) to assess the salience and degree of installation of various drivers by implementation stage, and an Implementation Capacity Analysis (ICA) measure to assess jurisdictions’ perceived improvements in key capacity building domains. The fourth paper highlights the work of one IC and compares and contrasts jurisdictions’ and providers’ perceptions of T/TA, which provides insights into the usefulness of TA. The symposium explores the complexity and practical challenges of attempting to rigorously evaluate technical assistance, limitations of the evaluation, and lessons learned in hindsight.

The proposed symposium is of importance and significance to the child welfare research field, particularly those studying T/TA. The evaluation incorporated a unique method for operationalizing, characterizing, and measuring T/TA and its outcomes. Gaining greater understanding of the differences in perceptions between providers and recipients of TA offers valuable insights into how providers may improve TA effectiveness. In addition, aggregate data on T/TA needs, the organizational and systems changes occurring within child welfare systems, and the barriers and facilitators to change provide a snapshot of the current status of the field which can be informative in setting a future social work research agenda.

* noted as presenting author
What Is Technical Assistance? Creating Descriptive Categories and Operational Definitions of “Training and Technical Assistance” for Quantitative Analysis
Jing Sun, MA, ICF International; Janet Griffith, PhD, ICF International; Joanna DeWolfe, MA, James Bell Associates; Rupinder Randhawa, MS, ICF International
Why One Method Won't Work: A Mixed-Method, Longitudinal Approach to Evaluating Training and Technical Assistance (T/TA)
James DeSantis, PhD, James Bell Associates; Pirkko Ahonen, PhD, James Bell Associates; Anita P. Barbee, PhD, University of Louisville
Coping with Complexity: Attempting to Measure and Compare Implementation Processes and Capacity Across Sites
Megan E. Fitzgerald, PhD, James Bell Associates; Jill Sanclimenti, MBA, ICF International; Mary I. Armstrong, PhD, University of South Florida; Michelle Graef, Phd, University of Nebraska, Lincoln
Technical Assistance: A Comparison Between Providers and Recipients
Berenice Rushovich, MSW, University of Maryland at Baltimore; Rochon Steward, MSW, University of Maryland at Baltimore; Leah Bartley, MSW, University of Maryland at Baltimore; Charlotte Lyn Bright, PhD, MSW, University of Maryland at Baltimore
See more of: Symposia