Methods: Based on methods developed by Mansell et al. (2011), data were drawn from Family and Children’s Services of the Waterloo Region’s client database to examine the first referral or ‘spell’ per family between 2012 and 2017 (N = 14,163). Each case was tracked for 12 months to investigate the sensitivity and specificity of the referral disposition decision. Four groups were derived from this analysis: True Positive (screen in and verified within 12 months); True Negative (screen out and not verified within 12 months); False Positive (Screen in and not verified); False Negative (screen out and verified within 12 months). Bivariate analyses were used between the four groups and the decision to investigate. For False Positives only, additional analyses were conducted to explore correlates of these outcomes. File reviews were also completed using random selection per year.
Results: While the sensitivity of the screening decision is high (Se=93%), the specificity is low (Sp=43%). Of cases investigated and substantiated only 36% of cases were transferred for ongoing services. Factors associated with false positives include referrals from schools and allegations of physical abuse. 25% of all false positive cases involved inappropriate discipline, 20% related to a caregiver risks, and 14% involved exposure to IPV.
Conclusions and Implications: The analysis allowed the organization to rethink the response to families who present as lower risk or who do not present with imminent safety concerns. More than 8,000 families over the five-year period were the subject of an intrusive investigation without any services or supports being offered. The fear and anxiety regarding false negative decisions appeared to be driving the organization's risk threshold down, yet only 2.7% of cases fell within this false negative grouping. The results of this analysis were used to facilitate discussions at an organizational level about whether the risk threshold and service responses fit the presenting issues faced by families referred to the organization. The study recommended that the service response provided to the profile of families within the false positive group should look substantively different from an intrusive child welfare investigation. As a result, the percentage of cases disposed to investigation dropped from 80% to 60% over the 12-month period.