Measuring the Impact of Child Maltreatment Prevention: Findings from the Retrospective Data Analysis of the Texas Child Maltreatment Programs

Background: Measuring the impact of child abuse and neglect prevention programs is challenging, particularly in selecting outcome measures that will truly measure the impact of the prevention program. Acts of child maltreatment are relatively rare events, and thus, substantiated reports often have a low occurrence rate over short evaluation time periods and small sample sizes.  Maltreatment prevention programs in Texas have traditionally tracked family involvement with child protective services only while the family received prevention services.  Thus, the long-term effectiveness of prevention services was largely unknown. This study utilizes seven years of data examining Texas’ child maltreatment programs. 

Methods: Data and sample: Data from the state of Texas Prevention and Early Intervention database was matched with the child maltreatment databased.  Any family who received prevention services and had a confirmed case of child maltreatment between 2008 to 2015 was used for this evaluation. The sample consists of the primary caregiver from 137,068 families and over 162,000 total observations (22,378 primary caregivers appear in multiple years).

The primary outcome measure of interest is the presence of a confirmed case of child maltreatment.  A descriptive analysis was conducted to first obtain the number of families who received prevention services who later had a substantiated case of maltreatment.  Additionally, two measures were tested with logistic regression to understand their ability to predict maltreatment. The Protective Factors Survey was completed by every family receiving prevention services and a risk assessment was conducted by whatever worker was administering services to the family.  Demographic variables include age, income, and marital status, county and zip code of residence.

Results: Only 3% of families receiving prevention services later had a confirmed child maltreatment case during this timeframe.  Significant differences in the likelihood of a confirmed case of child maltreatment by primary caregiver age, income, marital status, and county of residence were also indicated. Evaluation results suggest that the type of service provided is related to meaningful differences in the likelihood of a confirmed case of child maltreatment, but that Protective Factor Survey scores (including pre-test, post-test, and change scores) are not reliably related to the likelihood of a confirmed case of child maltreatment. Rather risk factors identified by caseworkers and combinations of demographic variables were stronger predictors of child maltreatment.

Conclusions: This study suggests that low proportions of families receiving prevention services have confirmed maltreatment cases.  However, given that the programs do not have a relevant comparison group, their effectiveness should be interpreted with caution.  In terms of measures, it appears that the Protective Factors Survey is not a strong predictor of maltreatment.  Rather, a worker-based assessment is needed to understand the risk of maltreatment.