Meta-analysis and systematic reviews

Meta-analysis is a statistical technique for synthesizing findings from multiple independent studies into a single quantitative estimate of effect size, typically reported as standardized mean difference (Cohen's d, Hedge's g) or odds ratios. Systematic reviews provide qualitative synthesis alongside meta-analysis, documenting search strategies, inclusion/exclusion criteria, and heterogeneity sources.

In misinformation and media-effects research, meta-analyses serve to aggregate findings across diverse methodologies and populations, estimate average effect sizes with confidence intervals, and identify moderators (variables) that explain when effects are stronger or weaker.

Key papers

• Walter et al. (2020) — Fact-Checking: A Meta-Analysis of What Works and for Whom — meta-analysis of 30 studies on fact-checking effectiveness; synthesizes findings on overall effect (d = 0.29), heterogeneity sources (motivated reasoning, ideology, message design, context), and publication bias.
• Ecker et al. (2022) — The psychological drivers of misinformation belief and its resistance to correction — comprehensive systematic review synthesizing cognitive, social, and affective mechanisms in false belief formation and interventions (pre-bunking, de-bunking); identifies evidence-based principles.

Methodological considerations

• Effect size standardization: Cohen's d allows comparison across studies using different measurement scales and sample sizes; heterogeneity (I² statistic) indicates whether variance exceeds chance expectations.
• Publication bias: smaller studies with null or weak effects may languish in file drawers; funnel plots and trim-and-fill procedures detect asymmetry.
• Moderator analysis: meta-regression identifies categorical or continuous variables (e.g., ideology, message length) that predict effect-size variation; allows hypothesis-testing about boundary conditions.
• Random-effects vs. fixed-effects models: fixed-effects assumes all studies estimate the same true effect; random-effects assumes effects vary across populations and contexts (preferred when heterogeneity is high).

Limitations

• Dependence on primary studies: meta-analyses inherit flaws from included studies (measurement error, design confounds, statistical power issues).
• Heterogeneity interpretation: high I² indicates effects vary substantially, but meta-analysis cannot always explain why without adequate coding of study features.
• Multiple comparisons: testing many moderators increases Type I error risk; pre-registration and sequential testing reduce but don't eliminate false discoveries.

Connections

• Fact-checking and corrections — Walter et al. meta-analysis synthesizes fact-checking intervention studies.
• Misinformation — meta-analyses synthesizing intervention effectiveness.