Placement Stability and Social-Emotional Well-Being: Examining Key Measures in a Formative Evaluation of Evidence-Based Parent Training for Children in Foster Care

Background/Purpose:  Following calls to expand parent training programs (Barth, 2009; Horwitz, et. al, 2010), many child welfare agencies have adopted parenting models from other fields. Despite evidence of their efficacy, more research is needed in child welfare to demonstrate conclusively whether these models positively impact safety, permanency, and well-being. This study examined key measures of placement stability and social-emotional well-being for a demonstration project of in-home Parent Management Training-Oregon model (PMTO) for children in foster care with serious emotional disturbance (SED). Extant research suggests that behavior problems affect placement stability and placement stability impacts social-emotional well-being (e.g., Rubin et al., 2007). Following a tollgate approach (Epstein & Klerman, 2012), this formative evaluation tested key measures prior to further investment in summative evaluation (Akin et al., under review). Specifically, the research aim was to observe whether measures of placement stability and social-emotional well-being performed as expected.

Methods:  Using a pretest-posttest randomized consent trial (Zelen, 1990), children identified as SED within six months of entering foster care were randomly assigned to PMTO or to a services-as-usual comparison group (N=121; PMTO group=78; comparison group=43). Child social-emotional well-being comprised three constructs (functioning, problem behavior, social skills) that were measured by the Child and Adolescent Functional Assessment Scale (CAFAS) (Hodges, 2004) and the Social Skills Improvement System (SSIS) (Gresham & Elliott, 1990) at baseline and 6-month post-test. Placement stability was calculated as an annualized rate of placement settings. A multi-group structural equation model was tested to observe the relationship of baseline well-being and placement stability on post-test well-being.

Results: Our final structural model indicated close or exact fit to the data (RMSEA=.000, CFI=1.00, SRMR=.043). Baseline measures of well-being did not significantly predict placement stability for either group, indicating that observed variation in placement stability was a function of mechanisms other than well-being. However, post-test well-being was significantly predicted by baseline well-being for both groups and, importantly, only the intervention group demonstrated significant effects of placement stability on post-test well-being. For the intervention group, as placement stability increased, post-test functioning and behavior significantly improved, demonstrating an association between well-being and placement stability that was not evident in the comparison group. Additionally, initial well-being significantly predicted later functioning and behavior for both study groups. Overall, well-being measures performed as expected and detected relationships between variables and variation among the groups as hypothesized. Additionally, placement stability may have been mediated by the intervention.

Conclusion: This study contributes to a growing literature on phased evaluations, which explicitly test newly-implemented interventions and corresponding data collection methods prior to investing in larger, resource-intensive summative evaluation. Formative results show that these well-being measures should provide adequate tests of hypotheses when applied to summative data. Although larger samples and distal outcomes are necessary for making conclusions about PMTO’s effectiveness for children in foster care, this study showed significant relationships between child well-being and placement stability for the PMTO group but not the comparison group. In combination with other formative testing, a summative evaluation is the next appropriate step to examine the intervention’s effectiveness.