Evidence Building and Information Accumulation: Bayesian Paradigm Cohesive for Child Welfare Intervention Research

Abstract: The dearth of replicable support for many programs that purport to be effective at improving child and family outcomes is eroding public confidence in the child welfare system's capacity to deliver on its promises of child safety, family permanence, and adolescent wellbeing. Now that the U.S. Congress has codified evidence standards for the funding of child welfare preventive services under the Family First Prevention Services Act (FFPSA), there is a pressing need for child welfare practitioners and researchers to conduct routine low-cost, rigorous evaluations that apply the Bayesian paradigm as a supplement to more conventional frequentist approaches to testing statistical conclusion validity. The frequentist approach ignores prior knowledge on the effects of interventions and, because analyses are not conditioned on prior information, there is a loss of relevant information about the promising effects of interventions designed to promote the safety and well-being of children in nurturing, permanent homes. This “isolationist” approach is in sharp contrast to decision-making and policymaking in child welfare and other areas that are generally cumulative and contingent on how well previous interventions, programs, and policies faired. A logical extension of the frequentist approach is to incorporate prior knowledge into analyses by using Bayesian modeling.

This workshop includes a panel of researchers with expertise in applied experimental research in child welfare, advanced statistical analyses, and child welfare policy. The panelists will discuss how the integration of Bayesian methods into phased-based approaches to evidence building holds great promise for policymaking by building on low-cost formative studies to generate replicable results in summative studies for what works, for whom, and under what conditions. Using examples from a low-cost, randomized control trial called Safe Families for Children (SFFC), this workshop provides insights into the Bayesian paradigm regarding its ability to build on or strengthen conclusions about the efficacy of a promising intervention for a vulnerable population: children at risk of entering foster care. The panel will demonstrate the advantages of the Bayesian paradigm and offer perspectives on future directions. After the workshop, participants will have an understanding of the core features, assumptions, methods, and challenges in using Bayesian modeling in intervention research.

Objectives: This workshop will: • Provide an overview of evidence-building and information accumulation using a low-cost, randomized control trial Safe Families for Children (SFFC) in child welfare research; • Introduce and illustrate Bayesian modeling, which incorporates prior knowledge in the data likelihood function; • Compare the frequentist and Bayesian analytic approaches; and • Demonstrate the use of Bayesian analytics in an application. By the end of the symposium, participants will be able to describe: • The key scientific rationale for Bayesian models in intervention research; • The advantages and limitations of Bayesian modeling; and • Steps in applying Bayesian models with intervention data.

Methods and Approach: PowerPoint presentation with handouts, lecture interspersed with questions and answers, software demonstration, and discussion.