Reporting Quality of Complex Social Intervention Trials: A Consort Extension for Complex Social Interventions

Purpose: Many interventions in social work and related fields are complex. Understanding trials that evaluate these complex social interventions requires detailed reports of the interventions tested and the methods used to evaluate them; however, several studies suggest that trial reports often omit important information. Inferior reporting of trials hinders proper critical appraisal of these studies, thereby impeding the effective transfer of research evidence to policy and practice decision-making. The CONSORT (CONsolidated Standards for Reporting Trials) guidelines have significantly improved trial reports in biomedicine, but the impact of guidelines in social work and related fields is unclear. The objective of this study is to demonstrate the need for and discuss the development of a new guideline to assist reporting of complex social intervention trials. This study has two research questions: (a) what is the extent of current guidance for reporting complex social intervention trials, and (b) what is the current quality of trial reports?

Methods: Two studies were conducted to answer the research questions. In Study 1, a systematic electronic literature search sought to identify all peer-reviewed reporting guidelines related to complex social intervention trials. These guidelines were assessed via frequency tabulations of adherence to best-practices in reporting guideline development and dissemination. All reporting standards found within these documents were then combined into a comprehensive checklist of standards for reporting complex social intervention trials. In Study 2, a systematic review of the reporting quality of complex social intervention trials was conducted. We included all complex social intervention trials published in 2010 in the 10 highest-impact factor journals in four key intervention disciplines (i.e., social work, clinical psychology, criminology, and education). Complex social intervention trials were assessed via frequency tabulations of adherence to a comprehensive list of standards from Study 1.

Results: Study 1 yielded 19 reporting guideline documents related to complex social intervention research, with a total of 147 reporting items. CONSORT guidelines demonstrated adherence to a higher percentage of best-practices for guideline development and dissemination than did guidelines developed by social and behavioural scientists. However, social and behavioural science guidelines yielded 89 modifications to reporting standards for complex social intervention trials that were not found in CONSORT guidelines. Study 2 yielded 239 complex social intervention trials. On average, reports adhered to less than 50% of reporting standards, and, importantly, trials in social work adhered to slightly less standards (39.5%) than the overall sample (41.9%).

Implications: Current reporting guidelines in the social and behavioural sciences have not been as rigorously developed and disseminated compared to CONSORT guidelines, yet the CONSORT guidelines do not provide sufficiently tailored reporting standards for complex social intervention trials, such as those in social work. Moreover, the reporting quality of these trials is suboptimal for integrating research into practice. Taken together, these results indicate that a new CONSORT Extension for Complex Social Intervention trials is needed. A project plan for developing this new extension according to best practices in guideline development will be discussed.