Assessing Children's Risk of Abuse: The Influence of Context and Worker Variables on Professional Judgment

Purpose: In an effort to improve the ability of child welfare agencies and individual workers within these agencies to accurately identify children at risk of harm, child welfare services throughout the world are moving towards highly standardized risk assessment tools. The assumption of these models is that uniform measures and training will result in consistent and accurate assessments. However, the importance assigned to standardized tools in child welfare practice has sparked considerable controversy due, in part, to questionable reliability and validity of the measures. Previous attempts to examine reliability and validity have focused primarily on retrospective reviews and paper vignette studies. What remains unclear, however, is the degree to which specific context and worker variables may influence professional judgment and the manner in which a worker assesses risk, even on a “standardized” instrument. This study uses both standardized risk assessment measures and standardized clients (professional actors) to investigate whether the previous experiences and pre-existing emotional and physiological state of child welfare workers interact with case-specific information and, in turn, influence professional judgment regarding the acute risk of child abuse.

Method: An experimental design was used to evaluate the judgment of risk among a sample of 94 workers recruited from nine child welfare agencies. All workers were presented with two simulated clinical interviews involving typical child welfare intake cases. Parents were portrayed by specially trained standardized patients. Using a systematic assignment process that approximated a random schedule, these actors alternately varied their level of aggression within the context of the scenario. Thus, each caseworker was exposed to a more confrontational client and a more cooperative client. Measures included pre-scenario administration of standardized instruments addressing prior trauma exposure and current PTSD symptoms. After exposure to interviews with standardized patients, workers completed two standardized risk assessment measures. One measure was a consensus-based risk assessment instrument, the other an actuarial tool.

Results: Workers had been exposed to a variety of critical events in the workplace including child death due to accident (29.2%) or maltreatment (13.5%), threats towards themselves (61.5%) and assaults against themselves (20.8%). Levels of traumatic stress were measured with the Impact of Event Scale–Revised, 12.5% of workers scored in the high range and 18.8% scored in the severe range of symptoms. There was considerable variation in assessment of risk on both scales, despite the fact that both scenarios were deliberately designed to reflect moderate risk. However, workers were more likely to correctly rate one of the scenarios using the actuarial tool (70%) than the consensus tool (40%). There were some significant differences in scores when scenarios were confrontational versus non-confrontational, but no clear pattern emerged. Post-traumatic stress symptoms were not generally associated with perceptions of risk.

Conclusion: This study found considerably variability on assessment of risk among 94 workers assessing two standardized clinical cases. Worker variables of trauma exposure and post-traumatic stress had limited impact on risk assessment. Confrontational parents were sometimes judged to be at higher risk than non-confrontational parents.