Research That Matters (January 17 - 20, 2008)

Background and Purpose: Systematic reviews and meta-analyses synthesize outcomes reported in articles or other research reports, and thus are able to answer certain theoretical and practical questions that primary studies cannot address (Cooper & Hedges, 1994; Petticrew & Roberts, 2006). Together, systematic reviews and meta-analyses, such as those published by the Cochrane and Campbell Collaborations, are used to examine the status of specific interventions used by helping professionals or the relative effectiveness of interventions for specific social problems (Gambrill, 2006), produce unbiased assessments of the state of knowledge on a given topic, map out areas of uncertainty, and identify direction for future research. Thus, systematic reviews and meta-analytic scholarship are important to the development and implementation of evidence-based models of practice.

Social workers are introduced to journal publications as a part of their academic preparation, and are therefore more likely to access systematic reviews and meta-analyses through peer-reviewed social work journals than through on-line systematic review databases such as Cochrane or Campbell. While the quality of systematic reviews and meta-analysis is carefully controlled by these two organizations, it is important to examine the quality of meta-analytic research in social work journals as we move into an era of evidence-based practice.

Methods: We hand-searched 38 high-prestige social work journals for meta-analytic research published from 1990 through 2006. A total of 37 articles were located. These articles were independently reviewed by the first two authors for quality of methods, reporting of results, and discussion of findings. Reliability of coding was assessed at .96, and disagreements were resolved by consensus.

Results: Several meta-analyses were published in journals not be easily identified as social work journals, despite classification by Sellers and associates (2004). Nearly one third of all meta-analysis articles were published by one journal, and 62% were published within just three journals. Most meta-analyses concerned the effectiveness of interventions, and half of the articles were senior-authored by social workers. The median number of studies analyzed was 50, and ranged from three to 305 primary studies. The methods reported by these articles were of uneven quality, with several critical omissions in reporting of methods and results. On the other hand, many of the articles used high quality methods and reporting.

Conclusions and Implications: In order to ensure that meta-analytic research in social work will be useful to practitioners, it is recommended that social work journals recruit reviewers familiar with meta-analytic procedure and adopt minimum standards for reporting of methods and results.

References

Cooper, H., & Hedges, L. V. (1994). The Handbook of research synthesis. New York: Sage.

Gambrill, E. (2006). Evidence-based practice and policy: Choices ahead. Research on Social Work Practice, 16, 338-357.

Petticrew, M. & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Malden, MA: Blackwell Publishing.

Sellers, S. L., Mathiesen, S. G., Perry, R., & Smith, T. (2004). Evaluation of social work journal quality: Citation versus reputation approaches. Journal of Social Work Education, 40, 143-160.