Methods: An on-going cluster randomized controlled trial (cRCT) has been employed to evaluate the long-term effectiveness of the Resilience Alliance and Peer Support Group intervention components in reducing STS. However, mixed methods have been used to evaluate implementation. Implementation surveys were administered across the state at 12- and 24-weeks to the 152 caseworkers and supervisors who were randomly assigned to the RA + PSG intervention. Valid responses for both time points were received from between 99 and 110 workers (the response rate varied by scale). The overall response rate was 72%. Surveys included the following four scales:
- Participant Satisfaction Scale, measuring the usefulness, practicality, and effectiveness of RA.
- Facilitator Working Alliance Survey, measuring the extent to which the facilitator was effective in delivering the RA curriculum.
- Group Climate Questionnaire; assesses dynamics among people in the RA groups (engagement and conflict subscales).
- Curry Transfer of Learning Scale; measuring extent to which participants feel they can apply knowledge learned through RA and levels of support for transfer of learning in the agency.
We ran paired t-tests to examine changes in summed scale scores between the 12- and 24-week measures for Resilience Alliance participants and we triangulated these results with qualitative comment data from participants.
Results: Caseworker and supervisor satisfaction with the delivery of Resilience Alliance dropped significantly between 12 and 24 weeks (t = -5.49, p < .0001), as did measures of Transfer of Learning (t = -5.80, p = .0001). However, there were no significant changes over the same time period in participant perceptions of facilitator effectiveness (t = -1.61, p < 0.110), nor for Group Engagement (t = -1.52, p = 0.131) or Group Conflict (t = 0.63, p = 0.533).
Conclusions and Implications: We conclude that drops in satisfaction and Transfer of Learning for RA participants may be explained by the rather repetitive nature of the Resilience Alliance curriculum. It is positive to note that group climate measures did not drop over the same period. This is important for the success of the Peer Support Groups which follow the RA intervention. Our presentation will dig into additional qualitative data sources documenting participant reaction to the RA curriculum and the Peer Support Groups to expand on these conclusions.