Estimates of Variance Components in Random Effects Meta-analysis: Sensitivity to Violations of Normality and Variance Homogeneity

Meta-analysis is a statistical technique through which information obtained from a collection of independent studies is analyzed and synthesized, potentially providing a more complete representation of the phenomenon under investigation. Models for meta-analysis may be roughly divided into those based upon fixed effects and those based upon random effects (Field, 2001; Hedges, 1994; Hedges & Vevea, 1998; Raudenbush, 1994). The primary goal of meta-analysis is to obtain estimates of population effect sizes and confidence bands around those estimates. For example, recent research in the area of educational reform related to the National Science Foundation’s Systemic Initiatives (Kromrey et al., 2002) has investigated the use of effect sizes as indices of the effectiveness of systemic change. These efforts have been directed towards synthesizing student outcome differences across subject areas, grade levels, and years of participation. Because sample effect sizes obtained for a metaanalysis typically present different magnitudes of estimation error, weighted means and variances are used to obtain the estimates of population effect sizes and confidence bands. For fixed effects models, these weights are given by