Is it time for the Cochrane Collaboration to reconsider its meta-analysis methodology?

Meta-analysis is becoming more popular in the biomedical literature and of increasing importance to clinicians, policy makers, funding bodies, and researchers for synthesizing practice guidelines, grant justification, and making policy decisions. Meta-analysis involves a quantitative analysis of multiple study outcomes to reach conclusions regarding an intervention. Often these studies are different in their design and conduct. One method of meta-analysis endorsed by the Cochrane Collaboration that has raised significant controversy is the randomeffects model, which assumes that underlying effects vary across differing study populations. Often larger studies, even if not well designed and meticulously conducted, have more information value than smaller studies. The random effects model redistributes weights in one direction only—from big to small studies— without addressing variations in study estimate related to study quality and conduct. Finally, this estimator cannot be expected to have a variance structure that is different from that of the arithmetic mean when heterogeneity is large.