On Causal Inference and the Limits of Disproportionality As a Construct

Background and Purpose: With the developments in causal inference with observational data as a backdrop, the first paper considers what it means to make causal statements when discussing foster care placement rate differences linked to Black children and White children. We start with common missteps that affect causal reasoning, but move quickly to a rather simple question: given how we measure whether Black children and White children have different experiences with the foster care system, are the measures we use well suited to causal inference? Disproportionality and disparity are among the most widely used measures. Both are useful as the basis for making comparative observations; however, once the pivot to causal reasoning has happened, their probative value has to be reconsidered. In this paper, we show why this is an important question by deconstructing the conventional measures of Black/White differences into their components with the requirements for causal reasoning in mind.

Methods: When reviewing the research used to link racial and ethnic disparity in the use of foster care to causal explanations, four challenges need the attention of researchers and advocates alike. They are model specification, the fallacies of causal reasoning, ecological inference in child protection research, and identification. For each of the issues raised, we demonstrate the importance of aligning the question being asked with the measure being used. We also show how the failure to pay close attention to the measurement challenges undermines the role of science as a tool for achieving greater equity.

Results: The results are organized into three clusters. First, we draw on the distinction between proportions, probabilities, and rates to highlight how foster care placement differences are measured. Efforts to connect causes to differences in the experience of Black children and White children with the foster care system have to respect the research question behind each measure. Having established those distinctions, we use a thought experiment to show how proportions and rates are used to answer a fundamental policy question: if we invest in services to bring greater equity to the foster care system, which measure provides us with the clearest view of success. We show why disproportionality and disparity have limited probative value. We close the first paper with a measurement approach that overcomes the challenges we describe.

Conclusions and implications: Regardless of how one conceptualizes a more just child welfare system, no matter what the new system is called, the family and child-serving system that emerges from the next round of reforms will propose both new structures and new processes. Among other concerns, public policy involves choosing between the structures and processes that work as intended versus those that do not. If we have to choose, how will we know the difference? Using what measures? What evidence will we use to substantiate the claim that, the solutions we applied to the problem of placement disparity worked? In the first paper, we outline the issues and propose a remedy that gives policy-makers and social scientists a robust evaluation strategy.