Method. Our sample included all children born between 1994 and 1998 with a foster care placement(s) in Illinois by age 18. We measured the outcome and the predictors by longitudinally tracing foster youth and their family members (e.g., parents and siblings) within child welfare case records. We constructed 35 predictors based on foster youth’s case records and 12 predictors based on family members’ case records. We estimated age-specific models. In our models, an age-1 model used predictors measured from birth to the first date of age 1 for those in foster care or from birth to the date of entering foster care at age 1 to estimate the likelihood of impermanence among youth in foster care at age 1. We estimated each age-specific model from age 0 to age 17 in the same manner. We used the data pertaining to children born between 1994 and 1997 to train logistic regression models. We validated models on the data about children born in 1998. We used AUC scores and precision scores to assess model performance.
Results. The AUC score generally increased from the age-0 model (0.622) to the age-17 model (0.720) with some fluctuations. The age-specific models for ages 2-3, 7-8, and 12-17 had AUC scores above the minimum acceptable threshold (0.7). Despite acceptable AUC scores, the precision scores were low (≤ 10%) in models of young ages (≤ age 5) because a large proportion of negatives among young foster youth (≥ 92% for ≤ age 5) led to excessive false positives. True positives outnumbered false positives (i.e., precision > 50%) in age-13 and older models. After adding predictors from family members’ records, AUC scores increased by about 0.01 to 0.04 for age-10 or younger models while there was little improvement in AUC scores for age-11 or older models.
Discussion/Implications. We found that simple predictive models with a sensible number of predictors could make acceptable predictions even at young ages. We also found that additional information about family members improved models, especially at younger ages. The widely different precision scores by age suggest a need for different types of prevention approaches by age, which will be explored in this session.