Relative Efficiency of Using Summary Statistics in Meta-Analysis: Recent Advances

Background and Purpose With the need of evidence-based practice, there is a growing number of social work researchers seeking robust methods, such as meta-analysis, to synthesize research findings from a series of independent studies. The fundamental idea behind meta-analyses is that there is a common truth behind all conceptually similar scientific studies, but which has been measured with certain errors within individual studies. The objective then is to use statistical approaches to derive a pooled estimate closest to the unknown common truth based on how these errors are perceived. Classical meta-analysis focuses on synthesizing descriptive statistics. The recent advances in this area have extended the synthesis from descriptive statistics to results of multivariate models. In this regard, Lin & Zeng's (2010) study is an important theoretical work that proves the relative efficiency of using summary statistics versus individual-level data in meta-analysis. Using results of this proof, researchers now can conduct model-based meta-analysis by using summary statistics shown in publications without accessing to the original data. This workshop aims to offer a summarized review of the Lin & Zeng's study, describe procedures of conducting meta-analyses synthesizing results of complicated multivariate models, and demonstrate the application of such a method to a study that combines results generated by Cox regression to investigate gender difference on time-to-reunification data among published studies of foster care outcomes. Contents The workshop first describes the classical meta-analysis that combines means from published studies. It employs the so-called variance-known hierarchical linear model (Raudenbush & Bryk, 2002, chapter 7) and demonstrates how to synthesize average outcome differences from 19 studies that employed a common intervention. It then discusses the fundamental question facing all meta-analysis researchers: to combine results from a series of independent studies involving complicated models, whether accessing to the original datasets is necessary? The Lin & Zeng's study shows that, for all commonly used parametric and semiparametric models, there is no asymptotic efficiency gain by analyzing original data if the parameter of main interest has a common value across studies, the nuisance parameters have distinct values among studies, and the summary statistics are based on maximum likelihood. This important work implies that meta-analysis based on summary statistics of complicated models is statistically efficient, cost-effective, and practically feasible. Such an advantage is obvious due to the fact that protection of human subjects often prohibits investigators from releasing original data. Demonstration and Results The workshop concludes by reviewing software packages available for conducting meta-analysis of multivariate models. Using a child welfare example that studies the gender difference on time-to-reunification data, the workshop shows how to combine results of Cox regression to address a critical research question regarding foster care outcomes. Results show that the meta-analysis of Cox regression is statistically efficient, empirically effective, and helps address important research questions that requires additional efforts otherwise. Implications Social work researchers now can conduct meta-analysis to combine results generated by a range of complicated models, including linear regression, logistic regression, Cox regression, structural equation modeling, and the like.