Integrated Data System Analysis for the Design of a Pay-for-Success Intervention in Foster Care

Background: This study details the analysis undertaken to design a novel model of social intervention in the area of children and family services. Known as Social Investment Bonds or Pay-for-Success (PFS) contracts, these models of social policy intervention exhibit an innovative financing and contracting structure. As established in a PFS contract, investors provide up-front funding for a project. Only if successful outcomes are achieved, the government pays investors the contracted amount. PFS deals rely on research to define the target population, quantify the anticipated fiscal savings and evaluate the impact. In this paper, we report on the research related to the first county level PFS project in the US. The Cuyahoga County, OH PFS provided an evidence-based program of services to homeless families with children in foster care.

Methods: An initial exploratory analysis involved probabilistic matching of caregiver records of children in Out of Home Placement (OHP) with records from other service agencies, such as juvenile court, jail, and homeless services. We performed Cox Proportional Hazard Model analyses on the integrated data in order to identify risk factors associated with longer times in foster care.  This analysis was used to identify a high-cost target population and to estimate the potential cost savings due to intervention. We also designed the randomized experiment needed to estimate a treatment effect (reduction in OHP days) in the presence of censoring and potential attrition bias and to estimate the size of the sample needed to identify a change in OHP large enough to trigger payments by the County.

Results: The experience of homelessness was found for 8% of OHP entrants, (homeless spell median=137 days; mean= 321 days). The hazard model confirmed that OHP duration was longer for children when the family experienced a homeless event: 710 as compared to 442 days at the median. Thus, the duration ratio of families with a homelessness event to those without such event is 1.6 at the median. This difference persists at the 25^th percentile (1.5) and at the 75^th percentile (1.3).  Given these results and calculating that the average cost of an OHP day was $75, we estimated that the treatment for these families would be considered successful if reduced OHP days by 25%.

Since the randomization was to be implemented at the caregiver level, while the treatment effect was to be calculated at the child level, clustering within families had to be considered. We implemented a covariate adaptive randomization scheme to balance the sample on number of children to a homeless mother and time spent in OHP prior to the homeless spell. We estimated a minimum sample of 506 children equally split in treatment and control groups to detect a reduction of 25% in OHP days.

Implications: There is growing interest in PFS as an innovative model for improving social welfare. The analysis presented here illustrates the benefits and tensions involved with implementing a field experiment to test a social intervention, particularly when the resulting estimates are to translate into payments by the public sector.