Session: Methodological Advances in Systematic Reviews and Meta-Analyses: An Update on Research Synthesis Methods (Society for Social Work and Research 21st Annual Conference - Ensure Healthy Development for all Youth)

Using evidence to inform practice and policy is essential to the social work profession; however, as the amount of research published each year grows exponentially, using research to inform practice and policy decisions has become more complicated and challenging. Research synthesis, a rigorous approach to synthesizing evidence, has become an important technique to manage, integrate and summarize primary research. Indeed, policy makers and practitioners are increasingly relying on rigorous synthesis of evidence to inform decisions. As with primary research methods, methodological advances are occurring rapidly in the area of research synthesis methods, and social work researchers are contributing to these advances in substantial ways. The goal of this symposium is to present three studies addressing various methodological challenges to advance research synthesis methods to more effectively address social work practice and policy issues.

The first paper addresses the challenges in synthesizing the range of study designs commonly used in social work research: qualitative (e.g., phenomenology), quantitative (e.g., hierarchical linear modeling) and mixed-method designs (e.g., RCTs followed by focus group). Typically, research synthesists have avoided synthesizing studies across designs due to the methodological challenges inherent in doing so; however, the need to use and integrate evidence across studies designs is essential for answering the complex questions that arise in social work. This paper describes a meta-summary approach to summarizing quantitative, qualitative and mixed-method results from primary studies. This approach to synthesizing research will be illustrated with a meta-summary study of understanding the change process of Solution-Focused Brief Therapy.

The second paper addresses the challenges of handling multiple dependent effect sizes within primary studies. Most quantitative meta-analyses of randomized control trials’ data in social science disciplines fail to address the dependence found between effect sizes within primary studies. A failure to appropriately handle within-study dependence can bias standard error estimates and thus negatively influence resulting statistical inferences. Therefore, it is important to consider the best statistical methods for handling within-study dependence of multiple effect sizes. This study examined how meta-analyses have handled dependent effect sizes and the statistical limitations of ad hoc methods still commonly used. Alternative statistical methods for handling dependent effect sizes will be introduced.

The third paper addresses the benefits and limitations of a relatively new method for synthesizing evidence: the overview of syntheses. As a result of the increased production of syntheses, including systematic reviews, researchers have begun to synthesize the syntheses. The production and use of overviews are becoming more prevalent, and may offer advantages over traditional research syntheses; however, overviews are a relatively new and undeveloped synthesis method that pose unique methodological challenges and may be problematic. This study systematically examined the methodological quality and current state of overviews to provide further guidance for conducting overviews, and advance the evolution of overview methods. Serious deficiencies in the conduct and reporting of overviews were found and unique methodological and statistical challenges inherent to this method were identified. As a result of this study, we developed conduct and reporting guidelines for overviews which will be presented and discussed.