Comparing a Machine Learning Algorithm to Two Jails’ Practice-As-Usual for Identifying Suicide Risk

Background: Suicide is the leading cause of death among individuals incarcerated in jails (Noonan, 2014). Incarceration is associated with suicide mortality post-release (Lim, et al., 2012). Few jails effectively screen for suicide risk (Boudreaux & Horowitz, 2014) and individuals may be less forthcoming with mental health needs during booking. New machine learning models could be utilized to improve suicide risk screening at jail booking, if evidence shows efficacy in identifying suicide risk, when compared to the jails’ current practices. This pilot study asked if a suicide risk algorithm, validated in out-patient health care settings, was more effective at identifying suicide risk in the year after jail, compared to the jails’ practice-as-usual (PAU).

Methods: Two jails in one Midwestern state provided individual identifiers for every person booked during the 6-month period (9/2021-2/2022). Each jail also noted who screened positive for suicide risk using the jails’ (PAU). One jail used questions from the Columbia Suicide Severity Rating Scale, in combination with other questions. The second jail used a set of 10 suicide risk-related questions, developed in-house. Five years of pre-jail admission Medicaid encounter data (mental, physical, and pharmacy) was gathered from the State’s Medicaid Office. The Mental Health Research Network (MHRN) risk score (Simon, et al., 2018) used this data to assign a risk score for each individual. In addition, data on suicide attempts 1-year after jail release was gathered from the Medicaid encounter data. The jails’ PAU was then compared to the MHRN risk score using suicide attempts that occurred 30-, 90-, 180-, and 1-year after release. Below we only present the details for 90-days post-release.

Results: There were 3,823 individuals screened positive (N = 1,136) or negative (N = 2,687) for suicide risk using the jails’ practice-as-usual. The MHRN risk score was also calculated for each case. At 90-days post booking, there was a significant difference between the number of suicide attempts between the positive and negative PAU screens (𝜒²= 15.354, p = <.001, V = .063) as well as the positive and negative MHRN risk scores (𝜒²= 68.018, p = <.001, V = .133). Positive MHRN risk scores (n = 704) predicted a higher proportion of suicide attempts 90 days post-booking (Positive Predictive Value (PPV): 5.1%), compared to positive PAU screens (PPV: 2.8%). Additionally, the Positive Likelihood Ratio (LR+) for MHRN is 3.3 (95% CI: 2.6, 4.0), while for PAU, LR+ is 1.8 (95% CI: 1.4, 2.2). This indicates that the MHRN screening tool is significantly more effective than PAU (p-value < 0.05). Similar patterns were also found at 30-, 180-, and 365-days post-booking.

Conclusions: This study provides insight into the advantages of using machine learning models to improve suicide risk detection among a high-risk population of individuals who are detained in jail. Results reveal that the suicide risk algorithm performed better than both jails’ PAU screens 90 days post-booking. While these data support the use of machine learning models for suicide risk screening, the feasibility and implementation of this technology across jail systems need to be further explored.