Child Welfare Triage: Use of Screening Threshold Analysis to Evaluate Intake Decision-Making

Background/Purpose: Child welfare agencies across the U.S. investigate a high number of reports of maltreatment with low percentage substantiation and service provision levels. Investigations take time, resources, and involvement with families, yet maltreatment is largely not confirmed by these investigations. Intake screening decisions are generally viewed as low risk for decisional error and are subject to a range of criteria and systemic biases that emphasize risk aversion. That said, investigation based on referrals of families of color are disparate with regard to population, placing such families at risk of further system involvement. The primary goal of the study was to establish a baseline understanding of the screening accuracy of a Midwestern child welfare agency’s centralized intake unit.

Methods: A screening threshold analysis (STA) was utilized to measure accuracy of the intake decisions at the agency’s intake unit. All reports made from July 2016 to June 2018 were classified with respect to accuracy and a proxy classification applied based on a 180-day follow-up period. A 2X2 decisional outcomes matrix and receiver operator characteristics (ROC) curve was used to describe intake accuracy. We also regressed false positive and false negative rates against the intake screening rate using report level data aggregated to county level.

Results: With an overall screen in rate of 92.87%, we found an AUC of .558. While the intake unit quite accurately identified reports on which maltreatment was confirmed in the study (True Positive Rate = 94.6%), the unit was poor at correctly identifying reports on which maltreatment was ultimately not confirmed (True Negative Rate = 8.5%). When analyzed at the sub-jurisdictional level false positive rates were found to have strong a positive correlation with screening rate slightly stronger than the corresponding negative correlation of the false negative rate.

Conclusions/Implications: Our findings indicate that the agency’s intake system is risk averse, resulting in high false positive rates. This highlights concerns that agency resources may be inappropriately directed towards investigation and away from intervention. The baseline findings regarding decisional error can be translated through a Continuous Quality Improvement (CQI) process through selected and testable changes to address goals in the intake decision making system to reduce the true negative rate while maintaining the true positive rate.