Using Evidence to Revise Colorado Adult Protective Service's Assessment Procedures

Background and Purpose:With the support of the Elder Justice Act and funding from the Administration on Community Living, state social services that respond to issues of elder abuse, neglect, and exploitation are increasingly considering how to use evidence to guide service development. In 2014, Colorado’s Adult Protective Services implemented a statewide, electronic assessment tool that captures risk and safety data on older clients. The assessment tool, which was developed using practice knowledge, is designed to help guide service planning and monitor outcomes over time. In 2016, a research-practice partnership was implemented to validate and revise the assessment tool. This study describes the original assessment tool in terms of reliability, validity, feasibility, and utility. The aim of the study was to identify ways to improve the assessment data to facilitate care planning for individual cases, to structure data for future research endeavors, and to monitor statewide outcomes over time.

Methods:The assessment tool included 70 risk and safety factors across the subscales of physical functioning, environmental context, financial resources, mental health and cognition, medical issues, and support systems. Workers completed the assessment tool at baseline (within 30 days of opening a case) and as a follow-up, upon closing the case. A mixed methods design was used. First, a description of the reliability and validity of retrospective assessment data for one year (n = 1,581 cases) was completed. Analysis involved assessing measurement properties, including Cronbach’s alpha and factor analysis for baseline and follow-up assessment data. Then, 49 workers were surveyed to test inter-rater reliability of assessment ratings on case vignettes. The analysis involved testing Kappa scores between workers and testing Kappa scores with a ‘master’ scoring for the case vignettes. Third, focus groups were conducted across 5 regions of the state on the assessment tool’s utility, feasibility, and areas for improvement (n = 42 workers).

Results:  Findings indicated ways revisions could minimize repetition, reflect better the scope of practice for adult protective service workers, and calculate valid total scores for monitoring changes over time. From the preliminary analysis, the scores trend as expected with follow-up risk scores being lower than baseline risk scores. Similarly, follow-up safety scores are higher than baseline safety scores. Yet, the reliability analysis pointed to subscales with minimal agreement. Themes regarding areas for improvement include: (1) omitting risk factors that are not modifiable, not applicable, or repetitive; (2) rewording items for increased clarity; (3) separating mental health and cognition into two subscales; (4) adding a subscale specific to mistreatment occurrences; and (5) increasing training to decrease subjectivity. 

Conclusion: Implications can highlight how research-practice partnerships can be used to improve adult protective services to promote elder justice through an evidence-driven assessment process. The state of Colorado has incorporated revisions to the assessment tool based on these findings in January 2017, and ongoing research is underway using administrative data to see if the revisions are linked to improved validity and reliability.