Predicting Recurrent Maltreatment in Child Welfare Systems: The Potential for Neural Network Models

While actuarial risk assessment represents a promising method for the early identification of children most likely to be re-reported for abuse/neglect, low predictive accuracy has stymied efforts to reduce the risk of re-report by targeting preventive service delivery to dynamic (i.e., changeable) risk factors that drive the likelihood of future maltreatment. To improve the potential for a risk-based approach to preventive service delivery in child welfare systems, this study’s objective was to use neural network analyses to increase the predictive accuracy of risk assessment by forgoing the typically held assumption that the probability of repeat maltreatment is linearly related to its predictors. Two research questions were addressed: (1) Do neural network models more accurately predict the probability of repeat maltreatment as compared with linear models (i.e., multilevel logistic regression models)?  (2) Do neural network models estimate a nonlinear relationship between the probability of repeat maltreatment and various predictors?

A sample of 6,747 children (one randomly selected child per family) and their primary caregivers in a Midwestern metropolitan area were followed from their first-known report for abuse/neglect for a period of at least 7 years (observations ended for children who reached 18 years of age) across child welfare, income maintenance, medical, mental health, special education, and juvenile and criminal justice systems.  A common state-level identifier linked administrative records for children and their primary caregivers to capture re-reports of maltreatment in addition to dates of contact with and service receipt from each of the publically funded service systems above.  The probability of a first and second re-report was predicted and compared across neural network and multilevel binary logistic regression models for (1) physical/sexual abuse, (2) neglect, and (3) abuse/neglect. Predictors of maltreatment included child and primary caregiver characteristics and their access to services (i.e., the type and timing of service receipt).

Using a cut-off point of 0.500 and the AUC as a global measure of predictive validity, all neural network models more accurately classified children as being likely (i.e., at high risk) or unlikely (i.e., at low risk) of being re-reported for maltreatment as compared with the logistic regression models.  Specifically, the AUCs for the neural network vs. logistic regression models were as follows: first re-report abuse (0.788 vs. 0.718), first re-report neglect (0.811 vs. 0.748), first re-report abuse/neglect (0.783 vs. 0.755), second re-report abuse (0.917 vs. 0.779), second re-report neglect (0.841 vs. 0.801), second re-report abuse/neglect (0.845 vs. 0.791). The magnitude of AUC improvements varied across models as did increased predictive accuracy in sensitivity and specificity.  Strong evidence of nonlinearity included curvilinear and interaction effects as identified during post-hoc analyses of average marginal effect probability plots.

Given that neural network analyses (a) outperformed multilevel logistic regression models, and (b) identified substantial nonlinear effects, child welfare systems should consider the expanded use of neural network models for more accurate risk assessments with better sensitivity and specificity.  Neural networks may help build a risk-informed classification scheme that strengthens empirical links between the magnitude and drivers of risk and services to reduce risk.