Confounding By Withdrawal in Relapse-Prevention Randomized Controlled Trials of Psychiatric Drugs: An Analysis of the Published Literature (2000-2017)

Background and Purpose: The results of relapse-prevention randomized controlled trials (RCTs) of prescribed psychotropic drugs—which typically and abruptly discontinue or withdraw drug treatment from some participants who are switched to placebo, while continuing drug treatment in other participants—may be confounded by drugs’ withdrawal effects. As few relapse-prevention RCTs account for withdrawal effects, at present only indirect measures using published articles’ data allow testing for possible presence of withdrawal confounding. Despite concerns regularly raised about withdrawal confounding, no empirical analysis of relapse-prevention studies has investigated it to date. This study tests this possible presence by determining whether a non-trivial proportion of endpoint relapse risk (50% or more) is already present by post-withdrawal time-points (WTs) when withdrawal effects would be salient.

Methods: Drug class-specific WTs ranging from 4-12 weeks were derived from a review of the clinical drug literature. We hypothesized that if differences in primary outcomes between drug-discontinued (placebo-switched) and drug-maintained groups at the WT were ≥ 50% of the same differences at the RCT’s endpoint, then results suggest confounding due to (typically abrupt) withdrawal of subjects from the study drug. In eligible relapse-prevention RCTs of antidepressant, antipsychotic, and stimulant drugs from 2000 to 2017 (n=30) selected independently by each author from a systematic review of discontinuation-focused publications, differences in relapse risk or behavioral measures between drug-discontinued (placebo) and drug-maintained groups were examined by arithmetic comparison of mean (±SD) and 95% confidence intervals of group behavioral measure scores (using SAS) or graphical analysis of survival plots (using the GNU Image Manipulation Program). Authors of all studies with missing data were contacted.

Results: A single RCT (3.3%) controlled for withdrawal effects, while 14 (46.6%) provided data to test for the confound. In RCTs comparing mean group scores, 2 of 3 (66.7%) suggested a withdrawal confound. In RCTs employing survival analysis, 6 of 9 analyses (66.7%) in 8 studies suggested a confound using longer WTs, while 8 of 12 (66.7%) analyses in 13 studies did so using shorter WTs. Among all analyses suggestive of a confound, an average of 86.23% (range 58.7% to 148.0%) of endpoint placebo vs. maintenance group difference was present by the WT under longer withdrawal effects onset and duration assumptions, and 90.28% (range 51.5% to 183.3%) under shorter assumptions.

Conclusions and Implications: In two-thirds of RCTs providing sufficient data, symptom scores rated in recently drug-discontinued participants duplicate between 51% and 183% of relapse risk differences at study’s end between these participants and drug-maintained participants. Our results, while limited by the indirect measure necessarily used, support the suggestion that unaccounted drug withdrawal effects confound the results of relapse-prevention RCTs. Given drugs’ centrality in mental health treatment and the “gold standard” status accorded RCTs in evidence-based mental health practice and research, these results require replication attempts. Until RCTs begin to systematically account for drug discontinuation effects, practitioners, policymakers, reporters, and researchers should evaluate their results critically.