Friday, 14 January 2005 - 12:00 PM

This presentation is part of: Poster Session I

Evaluation of the Performances of Risk Assessment Instruments in Child Protective Services (CPS)

Purpose: "Risk assessment" refers to some types of structural and systematic measurements to assist caseworkers in decision-making regarding the risk of future child maltreatment (Gambrill & Slovinsky, 2001). According to Lyon, Doueck, and Wodarski (1996), performances of risk assessment instruments were reported as disappointing in general. In order to contribute to the establishment of evidence-based practice of risk assessment, the current study had three purposes: (1) to examine the reliability and validity of the current risk assessment instruments, as reported in published evaluations since 1996 and (2) to identify risk factors that showed predictability of future harms; (3) to investigate the status of the evaluation studies in terms of rigorousness of their methodologies.

Methodology: We searched evaluation studies that examined the psychometric properties of exiting risk assessment instruments. The search criteria were as follows: (1) a study must have been published after undergoing peer review process; (2) a study must have been conducted by independent researchers who were not affiliated with the agencies that used the models; (3) a study must have provided empirical evidence regarding psychometric properties of risk assessment instruments or risk factors.

Results: We found nine evaluation studies that investigated the level of reliability and/or validity of the six risk assessment models: (1) the Children At Risk Field (CARF) system, (2) the Washington Assessment of Risk Matrix (WARM), (3) the California Family Assessment Factor Analysis (CFAFA), (4) the Michigan Family Risk Assessment of Abuse and Neglect (FRAAN), (5) Illinois Child Endangerment Risk Assessment Protocol (CERAP), and (6) the Utah family assessment model. CARF system showed a proper level of reliability in making decisions regarding maltreatment allegations. However, the content validity of CARF system was reported as poor. FRAAN indicated a much higher level of reliability and predictive validity than CFAFA and WARM. In addition, WARM showed an inadequate level of construct validity. CERAP was reported to show a satisfactory level of predictive validity. Utah Risk Assessment Scales indicated adequate levels of internal consistency and inter-rater reliability, but a disappointing level of construct validity. Further, the current study results revealed that only three studies examined the predictability of risk factors included in existing risk assessment instruments. Moreover, the studied risk factors varied across the two studies. The current study also identified four types of methodological limitations of the reviewed evaluation studies: lack of uniformity in methodology, inability to differentiate harms (i.e. maltreatment types), lack of qualitative research, and underdevelopment of predictive risk factor constructs.

Implications for practice: Our study results suggested that greater research endeavors are needed to establish reliability and validity of risk assessment instruments. Furthermore, the instruments should overcome various methodological problems. In addition, the fact that none of reviewed studies evaluated cultural competency of existing risk assessment instruments raises a concern of practice accountability. Based on the study results, a caution is recommended in using risk assessment instruments in CPS decision-making processes until a sound empirical research base is established.

See more of Poster Session I
See more of Oral and Poster

See more of Celebrating a Decade of SSWR (January 13 - 16, 2005)