Engaging with Mental Health and Criminal Legal Systems Collaboration Stakeholders to Develop Process and Outcomes Measures

Background and Purpose: Despite significant efforts to reduce the number of people with mental illnesses in the criminal legal system, they continue to be overrepresented within jails, prisons, and probation/parole. Sequential Intercept Mapping (SIM) is a cross-systems approach to strengthening local strategies in identifying problem areas and implementing services to divert or reduce contact with the criminal legal system, yet few tools exist for communities to evaluate these efforts. The aim of this study was to collaboratively develop a self-evaluation measure that communities can use to document progress in achieving outcomes (systems- and individual-level) and to understand process goals (e.g., collaboration).

Methods: Systems collaboration leaders and experts by experience across one Midwestern state were recruited to participate in focus groups to understand attitudes, knowledge, and beliefs about cross-systems collaboration and to identify common and consequential barriers to measuring processes and outcomes related to collaborative efforts. These findings were integrated with existing planning and evaluation measures to develop a working self-evaluation tool. Participants were invited to then engage in co-design sessions to create and refine a range of tools aimed at overcoming the barriers to self-evaluation of collaborative processes and outcomes identified during focus groups. This research was approved by the IRB at Northeast Ohio Medical University (#22-007).

Results: Fifteen systems collaboration leaders and 10 experts by experience participated in at least one focus group or co-design session (N = 25). At the focus groups sessions, community members engaged in a discussion about the value of evaluating collaborative processes and outcomes, along with possible metrics to assess these concepts. One barrier for communities is in the lack of self-evaluation measures available to community-based stakeholders that focus on process goals (e.g., collaboration). Participants observed that collaborative groups often focus on the development, funding, and sustainability of new programming, rather than focusing on collaborative processes or the outcomes of these efforts. Co-design sessions provided an opportunity for collaboration members to creatively engage in developing useful tools to be incorporated into an overall measure.

Conclusions and Implications: Systems collaboration stakeholders are seldom asked about how they are conducting self-evaluation of their efforts, yet they are important contributors to building a useable self-evaluation measure. Building community capacity is as important as developing self-evaluation measures if the goal is to increase the impact of community collaborative efforts. Improving systems collaboration capacity for self-evaluation and integrating useable measures into this process, has the potential to reduce the number of individuals with mental illnesses involved in the criminal legal system.