Developing Instruments to Assess a NYC APS Caseworker Engagement Training

Background and Purpose

Elder abuse is significantly under-reported: a New York study suggests that for every known case, 24 go unreported. Key reasons for underreporting and difficulty substantiating abuse include older adults’ reticence to disclose, and professionals’ lack of training and skills in facilitating such disclosure.  Thus, one potential avenue for addressing underreporting is to train Adult Protective Services (APS) workers to engage clients in ways that maximize potential disclosure. Literature on relational approaches supports the view that attuned connection deepens trust and increases client empowerment.  Specialized training to increase APS workers’ knowledge and skills in attuned client engagement could therefore potentially lead to increased disclosure.  

However, a literature review revealed a dearth of research evaluating the impact of engagement training on APS workers, as well as a lack of instruments assessing training impact. Therefore, we developed and piloted several instruments to evaluate APS worker engagement and intervention knowledge and self-assessed confidence in engagement skills prior to and following a training designed to increase skills in facilitating older adults’ disclosure of abuse.

Methods

A multidisciplinary team of five combining expertise in andragogical theory, training techniques, assessment, gerontological social work and APS practice collaborated over a year to design assessment instruments for an 8-hour APS worker engagement training developed by the same team.  Issues addressed included discussion of best testing formats, types of learning assessed (verbal information domain, intellectual skills domain, and attitudinal domain), item sequencing, scoring procedures, identification of existing measures that could potentially be adapted, and evaluation of test items. Instruments were pilot-tested with three volunteer APS workers and further refined and assessed for face validity and reliability before beyond utilized in three separate iterations of the engagement training, with a total of 58 participants.

Results

The instrument design process yielded:

1. Pre/post-training instrument assessing knowledge of relational engagement.
2. Pre/and post-training instrument assessing self-confidence in engagement skills. 
3. Pre/post-training case template asking for description of a client interaction pre- and post-training and reflecting on if and how knowledge or skills learned in training were applied.
4. Focus group questions addressing strengths and weaknesses of training and skill sustainability.

Instruments provided both quantitative and qualitative information on APS workers’ knowledge gains, self-assessed confidence in applying material learned, self-assessed behavioral changes in assessment and engagement, and feedback on the training itself.  We present data on the quantitative measures’ internal consistency as well as strengths and limitations of the instruments.

Conclusions and Implications

We developed instruments that can be used in future studies of APS worker engagement.  While response rates for pre-assessment instruments were high, post-training response rate dropped significantly; focus groups indicated that participants were tired after the training and eager to leave. We also encountered low response rates on the post-training case reflection, despite efforts at follow-up, and relatively low participation in focus groups, both due to participants’ packed schedules. More research assessing worker behavior is needed. This work should address these logistical challenges while also involving APS workers in the development of assessments of training outcomes.