Evaluation of the Illinois Adoption Preservation and Linkages Program (APAL) Using a Regression Discontinuity Design

Background and Purpose: The number of children in U.S. foster care has decreased from over half a million in the mid-1990’s to slightly over 400,000 in recent years, at least partially due to increases in adoptions and legal guardianships of foster youth. However, evidence suggests that post-permanency families experience high service needs and long-term child well-being concerns (e.g., externalizing behavior problems), as well as troubling rates of placement instability. Approximately 2% to 15% of former foster youth experience placement changes after legal adoption or guardianship, and placement instability is associated with poor child outcomes related to behavior, mental health, and academic achievement. The purpose of this study was to evaluate the impact of the Illinois Adoption Preservation and Linkages intervention (APAL), a brief needs assessment and service referral program designed to improve placement stability and well-being for youth who exit foster care to permanency. It was hypothesized that APAL participation would be associated with fewer child behavior problems and higher caregiver commitment to the adoption or guardianship.

Methods:  Survey data was obtained by the Illinois Department of Child and Family Services (IDCFS) from 437 caretakers of former foster youth ages 12 to 17 years old who resided in legally permanent adoptive or guardianship homes in Illinois and was linked to IDCFS administrative data. A regression discontinuity (RD) design was implemented to estimate the Average Treatment Effects (ATEs) of APAL on two outcomes, child behavior problems (measured by the Behavior Problems Index) and caregiver commitment (measured with a multi-item survey  tool), because intervention assignment was completely determined by youth age, with four different assignment discontinuities built into the study design. Two average treatment effects were estimated for each outcome, a sharp ATE that assumed perfect compliance with treatment assignment, and a fuzzy ATEC that adjusted for the probability of treatment receipt and applied only to compliers. Bivariate statistical analyses were first conducted to test for treatment differences in child and parent characteristics, as well as outcomes, across intervention groups. Then, SRD and FRD block, or hierarchical, regression models were estimated to examine the relationship between APAL assignment and each outcome.  

Results: Results showed that APAL participation was associated with fewer child behavior problems, but not higher caregiver commitment.  No relationship was found between child age and behavior, but a nonlinear relationship was indicated between child age and caregiver commitment, with child age 16 the lowest point of caregiver commitment. Finally, guardianship as compared to adoption was associated with slightly worse outcomes on both outcomes, possibly due to self-selection effects, and more children in the home were associated with higher caregiver commitment.

Conclusions: This study has implications for child welfare intervention design and practice. Results support the use of a brief, targeted family support and referral program to improve caregiver perceptions of child behaviors in permanent adoptive and guardianship homes.  This study also has implications for research, because RD is a relatively under-used design in social work program evaluation, and requires fewer assumptions than most observational designs to identify treatment effects.