Obstacles to the Diffusion of Knowledge: Haphazard Reviews, Propaganda, and Pseudo-Science in the Professional Literature

It does little good for scholars to conduct high quality research to test the accuracy of assessment methods and the effectiveness of service programs--and it does little good for clients to participate in these studies--if results are not “taken up” properly. The purpose of this symposium is to illustrate concerns in the reporting and dissemination of empirical research in the published literature in social work and related fields. These include the failures of published research reviews to find, critically appraise, and summarize relevant empirical evidence using scientific methods and current standards for research reviews, along with concerns about the prevalence of propaganda and pseudo-science in the published literature. Ways to encourage effective knowledge diffusion are offered, drawing on related research. The roundtable includes three presentations with ample time for discussion.

The first presentation is titled “the hazards of haphazard reviews.” Recent publications in peer-reviewed journals (in social work, psychology, and related fields) suggest that many scholars are unaware of current standards for research synthesis (i.e., PRIMSA and MARS guidelines). Many reviewers use traditional, narrative approaches when reviewing results of empirical research. Traditional review methods were termed “haphazard” by Petticrew and Roberts (2006), because better methods are available, and haphazard reviews are vulnerable to many sources of bias. This presentation describes the various forms and consequences of haphazard reviews. Recent research on outcome reporting, publication, and dissemination biases is presented, along with evidence that haphazard reviews tend to distort evidence and lead to the wrong conclusions.

The second presentation concerns “professional literature as advertisement.” The similarities between advertisements and the published literature in relation to both goals (e.g. to sell a product) and methods (e.g., inflated claims, question begging, asserting what must be argued) will be described drawing on related research. Common grand narratives appealed to such as progress, cure, and self-control are identified and illustrated. Reasons are suggested for this striking overlap as well as remedies including use of methodological filters (e.g. PRISMA and CONSORT guidelines) and use of a Propaganda Index designed to detect the misleading framing of problems. Data will be presented regarding use of this Index (Gambrill & Reiman, 2010).

The third presentation describes “pseudo-science in social work.” Pseudo-science has the trappings of science without the substance. This presentation examines the promotion and use of interventions that have no real evidentiary support. Examples from the field of social work are given, their prevalence discussed, and possible remedies suggested. Characteristics of pseudo-scientific and quack treatments are considered, along with real examples of licensed social workers providing bogus services. A checklist with warning signs to help identify quack treatment is provided. The presenter will also discuss steps the profession can take to reduce the likelihood that well-intentioned practitioners will be seduced by pseudoscience.

Discussion will center on the kinds of obstacles identified by the presenters as well as the options they suggest for remedies.