How "Grand" Are the Grand Challenges? a Critical Review of the Evidence for Social Work's Grand Challenges Initiative

Background and Purpose

In 2012, leading social work scholars gathered to identify 12 Grand Challenges (GCs) for the field of Social Work.  The GCs offer an agenda for social workers that are interested in effecting social change related to major social problems e.g. poverty, income inequality, environmental issues, and violence. Researchers utilized the papers to identify innovative ways in which social workers could resolve these problems within 10 years.  This paper uses a narrative critical review to explore the evidence used to make these assertions.             

Study Objectives

To determine the rigor of the evidence for the 12 GCs, this content analysis was conducted.  The study sought to evaluate the studies based on two frameworks:  1) the increasing importance assigned to systematic reviews in informing effective social work practice (Shlonsky et al., 2011) and 2) the GC criteria established by the GC Committee itself,  specifically the GC addressing areas that “are amenable to measurable change within 10 years” and “that have boiled down the science to focus on the best work” to be applied by social workers (AASWSW, 2016).

Method

A directed content analysis approach  was employed to appraise the referenced intervention studies in the 12 Grand Challenges papers (total N=17).  Each GC paper was coded for the degree to which the paper cited the most rigorous quantitative and qualitative research designs relevant to the topic, how often key empirical studies cited had been replicated, and how often they cited research syntheses e.g. systematic reviews. All intervention studies were hand searched, appraised, and included in the coding process.  Two members of the team independently coded each Grand Challenge paper, eventually working together to establish an acceptable kappa of inter-rater reliability.

Results

Each Grand Challenge was assigned an overall score for its scientific rigor, ranging from 1 (low rigor) to 10 (high).  Findings from the 17 GC papers indicate that 12 Grand Challenges indicate that only 30% of the papers had a score of 8/10, indicating high scientific rigor for the evidentiary claims of the paper.  Few GC papers included a systematic review, and while rigorous research was cited, many of the studies hadn’t been replicated.  Almost none of the GC papers provided compelling scientific evidence that the problem being discussed could be successfully addressed in 10 years.

Implications

Findings indicate a number of concerns, as the Grand Challenges Initiative begins:

1)   a seeming lack of high-quality evidence argues against the Grand Challenge being accomplished within the 10-year framework outlined by the Initiative;

2)   without clear evidence e.g. systematic reviews supporting the interventions discussed, policymakers may question the benefit of adopting these ambitious plans;

3)   the presence of consensus statements related to the quality of evidence seems to run counter to the core ideas of evidence-informed practice

4)   The 12 Grand Challenges papers provide some insight on these issues, but without clear structure or guidelines they lack the rigor necessary to inform policy.  A more inclusive and rigorous approach to developing and discussing social work issues and problems is offered.