Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation

Contributions

• Replicability demonstration: Pre-registered meta-analysis of all 20 accuracy prompt experiments conducted by the authors' group 2017–2020, addressing publication bias and establishing robust effect with identical analytical procedure across studies.
• Generalization evidence: Shows accuracy prompt effects do not significantly differ across political headlines vs. COVID-19 headlines, gender, race, political ideology, education, or baseline sharing discernment—though effect is larger for older, more reflective, and more attentive participants.
• Mechanistic validation: Treatment effect on false-news sharing (−10% relative to control) is strongly correlated with headline perceived accuracy (r > 0.77), confirming that accuracy-prompt efficacy is calibrated to headlines' plausibility, not universal across content.
• Replicability across platforms: Effect holds on MTurk (Amazon Mechanical Turk), Lucid, and YouGov, with consistent meta-analytic estimate (Cochran's Q = 23.3, p = 0.253, suggesting no heterogeneity in effect across studies).

Method

Study selection: 20 experiments identified from the authors' internal records (2017–2020) that included an accuracy prompt condition and measured sharing outcomes. All experiments included regardless of results (addressing file-drawer bias). Total N = 26,863 participants across surveys and field trials.

Accuracy prompt variations tested: Six main types—Evaluation (rate accuracy of neutral headline), Importance norms (told important to share only accurate news), Tips (digital literacy tips), PSA video (30-second public service announcement), Reason (asked to think about accuracy in reasoned way), and variations combining multiple elements. Only conditions explicitly directing attention to accuracy were included.

Outcome measures:

• Sharing discernment: standardized difference in sharing likelihood between true and false headlines, comparing treatment and control. Primary metric used in meta-analysis.
• Sharing intentions for false news: direct measure of intent to share specific false headlines (absolute effect size −3.4% to −11% depending on study).
• Sharing intentions for true news: largely null (effect ≈ 0%; accuracy prompts do not increase unwanted sharing of true content).

Analysis approach:

• Random-effects meta-analysis pooling results across all 20 studies
• Examined heterogeneity by sample type (MTurk, Lucid, YouGov), headline content (politics vs. COVID-19), intervention modality (passive exposure vs. active evaluation)
• Subgroup analysis for moderating variables: gender, race, political ideology, education, baseline accuracy judgment, cognitive reflection test (CRT) scores
• Correlation analysis between headline perceived accuracy and treatment effect (study-level), supporting causal mechanism

Results

Primary finding: Accuracy prompts significantly increase sharing discernment across the 20 experiments (meta-analytic estimate: b = 0.038, z = 7.102, p < 0.001), translating to a 71.7% relative increase over baseline sharing discernment in control condition. This is reflected in the absolute reduction of ~10% in false-news sharing intentions relative to control across meta-analytic pooled estimate.

Sharing of false news: Accuracy prompts significantly decreased sharing intentions for false headlines (b = −0.034, z = −7.851, p < 0.001), approximately 11% reduction relative to baseline control sharing for false content.

Sharing of true news: Accuracy prompts did not significantly increase sharing intentions for true news (b = 0.009, z = 1.189, p = 0.059), confirming that the intervention improves selective sharing rather than increasing overall sharing or inducing extreme skepticism.

Robustness and generalization:

• No significant moderation by headline type: Effect on false-news sharing does not differ significantly between politics and COVID-19 headlines (Cochran's Q interaction = 0.012, p = 0.912).
• No moderation by demographic group: Gender, race, political ideology (conservative vs. liberal), and education level did not significantly moderate treatment effect. However, effect was significantly larger for older participants (more representative samples vs. MTurk), more reflective participants (higher cognitive reflection test scores), and more attentive participants (higher baseline accuracy judgment).
• Mechanistic support: Study-level correlation between headline perceived accuracy and treatment effect on sharing was strong (r > 0.77 for false headlines), indicating the intervention works by redirecting attention to accuracy rather than universal skepticism or ideology shifts.
• Effect persistence: Treatment effect on sharing discernment does not decay across trial repetitions (3-way interaction between trial number, headline veracity, and treatment was non-significant).

Within-subjects analysis: When participants rated accuracy of multiple headlines before sharing task, repeated rating (vs. single-question evaluation) amplified effect (100% relative increase in discernment vs. baseline single-question effect).

Notes

This paper is a rigorous meta-analytic assessment of accuracy prompts. The key strengths are: (1) the transparent, pre-registered analytical protocol applied uniformly across all 20 experiments; (2) the complete inclusion of all qualifying experiments regardless of outcome, directly addressing publication bias; (3) the mechanistic evidence linking treatment effect to headline plausibility (strong r correlation), supporting the attention-redirection theory over alternative explanations; and (4) evidence from multiple platforms (MTurk, Lucid, YouGov) showing the effect is not artifact of a single recruitment pool.

Limitations include: (1) the meta-analysis covers only a single research group's experiments, raising questions about whether effects generalize to accuracy prompts designed by other researchers or deployed by platforms; (2) while field experiments show the effect persists outside survey contexts, the absolute effect size in field settings (~10% reduction) is smaller than survey experiments (~36% increase in discernment Cohen's d), suggesting real-world impact may be more modest; (3) all studies are from U.S. samples, limiting cross-cultural generalizability; (4) longest follow-up is 24 hours for Twitter field trial, leaving open questions about persistence over weeks or months.

The paper makes a compelling case for accuracy prompts as a scalable, low-cost intervention, but emphasizes they are one component of a multi-level approach (platform design, media literacy, fact-checking) rather than a complete solution to misinformation. The mechanistic insight—that misinformation sharing stems partly from inattention rather than lack of ability to distinguish truth from falsehood—remains the paper's most important contribution for intervention design.