The Psychometric Properties of the Outcome Rating Scale and Session Rating Scale Used in Psychotherapy

Background: The Outcome Rating Scale (ORS) and Session Rating Scale (SRS) are two commonly used measures in psychotherapy practice. Due to increasing demands by managed care companies, evidence of outcomes following treatment has been increasingly emphasized. Outcome measures allow therapists to systematically measure client progress through the ongoing collection of data.

Description of Problem: Since the ORS and SRS are designed to be used in clinical practice, demonstration of reliability and validity is ideal with clinical populations instead of non-clinical populations. To our knowledge, there have been no reviews evaluating the overall psychometric properties of the ORS and SRS in clinical populations.  

Purpose: In order for these outcome measures to have utility in practice, review of demonstrated psychometric properties is essential. This narrative review discusses and summarizes the results of studies that have evaluated the reliability and validity of the ORS and SRS in clinical populations.

Research Questions: What is known about the reliability of the ORS with adult clinical samples? What is known about the validity of the ORS with adult clinical samples? What is known about the reliability of the SRS with adult clinical samples? What is known about the validity of the SRS with adult clinical samples?

Methods: We conducted and electronic search of two databases for articles that met our inclusion criteria. Reference mining and searching the websites of the creators of the scales were two other strategies used. We reviewed 275 studies to assess whether they met our inclusion criteria and of these, 263 articles were excluded. A total of 11 studies met our inclusion criteria in this review.

Summary of Findings: The ORS demonstrated moderate to high reliability (n=9) as assessed by Cronbach alpha (.82 - .92) and low to moderate test-retest reliability (n=3) scores (.51). Concurrent validity findings (n=2) for the ORS showed moderate correlations with the longer outcome measure (OQ45), moderate correlations with depression, anxiety and stress using the DASS, moderate correlations with self-esteem using RESS, moderate to high correlations with quality of life using QOLS, and moderate correlations with depression using the PHQ-9. The SRS showed high reliability (n=7) using Cronbach alpha (.88-.97) and low to moderate test retest reliability (n=3) (.64 - .66). Concurrent validity of the SRS (n=3) with similar measures of therapeutic alliance, the HAQ II and the WAI, was lower than anticipated.

Discussion: When selecting outcome measures, clinicians should be aware that the ORS is a measure of overall distress and may not capture certain clinical constructs. Regarding the SRS, clinicians should be aware of limitations in construct validity with longer measures. Future research efforts should focus on validation of the ORS and SRS within larger, more diverse samples and specific clinical populations with a wider range of symptom severity.