The Evidence for Risk Assessment Tools Used in CPS Investigations: A Scoping Review

Background and Purpose. Risk assessment tools are commonly used by CPS agencies to augment decision-making about alleged child maltreatment. The design and use of these tools are evolving alongside technological advances that offer new approaches to using administrative data to evaluate risk factors in a household. Additionally, risk assessment tools are an essential component of the implementation of the FFPSA legislation. As such, conversations about risk assessment tools have shifted in focus such that a compilation and reevaluation of the literature is warranted. In this scoping review, we comprehensively surveyed the quantitative literature evaluating tools used by CPS agencies to measure risk and the evidence that may support (or not support) the validity and reliability of these tools both generally and in demographic subgroups.

Methods. We conducted a scoping review because it is a flexible approach that can be used to systematically map evidence when the relevant body of literature is wide ranging and heterogeneous but addresses a specific, practice-oriented question (Arksey & O’Malley, 2005; Levac et al., 2010). We developed a protocol in alignment with PRISMA-ScR (Tricco et al., 2018). Included research had to be a quantitative evaluation of a risk assessment tool used by a CPS agency and published between 1990 and May 2021. We used a multiphase selective approach to screening with at least two screeners. In total, 2,155 unique studies were subjected to double-independent title and abstract screening and 160 studies were subject to full-text review. A standardized data extraction spreadsheet was used to collect information from supporting studies.

Results. A final sample of 25 studies was included for review and data extraction: 21 peer-reviewed journal articles, two doctoral dissertations, and two reports. We present an overview of the consensus, actuarial, and automated algorithmic risk assessment tools included in the studies. We then summarize the methods used to assess validity and reliability and review the conceptual dimensions of risk included in each tool. Much of the literature focused on relative validity and reliability—specifically, comparisons between actuarial and consensus-based tools. With few exceptions, there is a dearth of evidence that tools are equally predictive across subgroups.

Conclusions and Implications. Much of the literature about risk assessment tool validation is dated and has not been updated, which may limit the ability to generalize this already dubious evidence of effectiveness to current practice conditions. Several studies are described as pilot or preliminary studies, though no further research confirmed the findings. Studies that assess tool validity and reliability are heterogeneous in study design, suggesting a lack of agreement about how to assess tools. The field should clarify objectives for the use of risk assessment tools and establish consensus around evidence standards. Agencies should be cautious about overreliance on tools for which evidence is limited.