Reducing misinformation sharing at scale using digital accuracy prompt ads

Contributions

• First field evidence at scale: First randomized controlled trials testing whether accuracy prompts delivered as digital advertisements can reduce misinformation sharing at scale (Facebook 33M users, Twitter 75K users), filling gap between survey experiments and real-world efficacy.
• Two-platform replication: Independent experiments on Facebook (industry-run) and Twitter (academic-run) show consistent direction of effects, supporting generalizability across platforms and implementation approaches.
• Content-neutral mechanism: Accuracy prompts work without identifying misinformation or fact-checking specific claims; instead they shift user attention to accuracy consideration before sharing—a mechanism compatible with privacy-preserving platform architectures and end-to-end encryption.
• Heterogeneous effects: On Facebook, effects concentrated among users who had recently shared misinformation (2.6% reduction), not among general audience. On Twitter, effects robust across multiple population segments with largest effects among active sharers.
• Mechanism evidence: Post-experimental surveys suggest treatment increases users' accuracy-focused reasoning about posts they encounter, consistent with attention-redirection theory rather than changed preferences or perceived threat.

Method

Facebook study design:

• Participants: 33 million Facebook users, two audience segments: (1) general audience reached through ads auction (~32M), (2) users identified as having repeatedly shared links flagged as misinformation in prior week (~1M, targeted for higher expected effect).
• Randomization: Users randomly assigned to treatment (accuracy prompt ads) or control (standard Facebook ads).
• Intervention: Treatment group received on average 3.2 accuracy prompt ads over 3 weeks. Ads presented in multiple formats:
• Static images: "When you share a post, think carefully before sharing. How important is it that the news you read online is accurate?" with headline evaluation framing
• 9-second video: emphasizing emotional language and encouraging accuracy checking
• Randomized selection: on some ad placements, users saw ad variants selected at random (poll, poll, or "How important is accuracy when sharing")
• Outcome measurement: Posts identified as containing misinformation using two approaches: (1) third-party fact-checkers (Facebook's fact-checking ecosystem), (2) domain-level quality classifiers trained to flag likely misinformation.
• Analysis period: Primary outcome measured in 60-minute window after first ad delivery. Secondary outcome: full 3-week campaign period.

Twitter study design:

• Participants: ~75,763 users across four separate experiments targeting different populations:
• Experiment 1: Highly active misinformation sharers (N~18K), identified by recent links to low-quality news sites
• Experiment 2: Highly active misinformation sharers, different sample (N~18K)
• Experiment 3: Users identified via deep-fake conspiracy hashtags (Canadian anti-vaccination protest)
• Experiment 4: Inactive misinformation sharers (low recent history of sharing low-quality content)
• Randomization: Block randomization by users' pre-treatment sharing levels; users assigned to treatment or control condition.
• Intervention: Treatment group exposed to average 2.91 accuracy prompt ads per day for 8+ days. Ads featured diverse creatives (50 different videos with varied framings) to prevent banner blindness:
• "Is the information accurate?" with different visual styles
• "Track your share record"
• Accuracy-importance framing ("88% of Americans believe it is important...")
• Outcome measurement: Quasi-Poisson regression models predicting number of low-quality posts shared per user per day. Low-quality content defined as retweets containing links to domains rated ≤0.70 on 0–1 quality scale (robust to alternative thresholds).
• Analysis: Fixed-effects models with clustering by user block and date effects.

Results

Facebook:

• 60-minute window: Treatment significantly reduced probability of sharing likely misinformation (1.8% reduction relative to control; b = −0.38, 95% CI [−0.69, −0.07], p = 0.018). Interaction effect: treatment effect 2.6% larger among users who had shared misinformation in week prior (b = −5.68, 95% CI [−9.60, −1.77], p = 0.004), indicating effects concentrated on target population.
• Full 3-week campaign: No significant effect on overall misinformation sharing over 3 weeks (b = −1.20, 95% CI [−3.60, 1.20], p = 0.313). Authors interpret this as evidence that periodic ad exposure insufficient for sustained behavior change; suggests more consistent messaging needed for durable effects.
• Specificity: Treatment effect specific to misinformation, not reducing engagement with accurate content (b = −0.0059, 95% CI [−0.0145, 0.0027], p = 0.179). Treatment did not reduce overall news sharing, supporting theoretical prediction that accuracy prompts redirect sharing to higher-quality content rather than suppressing engagement broadly.
• Moderators: No significant interactions by age (>65), gender, education, or political ideology, suggesting effects generalize across demographics.

Twitter:

• Active misinformation sharers (Experiments 1–3, meta-analysis): Overall treatment effect: 3.7% reduction in low-quality content sharing (b = −0.038, 95% CI [−0.062, −0.013], p = 0.002). Larger effect among users who actually viewed at least one ad (average treatment effect on treated, ATT): 6.3% reduction (b = −0.065, 95% CI [−0.107, −0.021], p = 0.002). Effect did not decay over 8–22 days (no significant treatment × day interaction), suggesting potential for sustained effects with repeated exposure.
• Inactive sharers (Experiment 4): No significant effect (ITT: b = 0.016, 95% CI [−0.006, 0.038], p = 0.154; ATT: b = 0.026, 95% CI [−0.010, 0.063], p = 0.158). Interpretation: effects contingent on user's prior engagement with low-quality content; intervention works for those already sharing misinformation.
• Specificity: Treatment effects specific to low-quality content (hashtag-based experiment documents reduction in conspiracy-related hashtag sharing). No effect on non-misinformation tweets (b = −0.004, 95% CI [−0.025, 0.018], p = 0.746), consistent with accuracy-focus mechanism.
• Moderators: No significant interactions by pre-treatment misinformation sharing level, political followers, or follower count. Effects heterogeneous across three experiments (Q(2) = 10.27, p = 0.010), suggesting context-dependence on population type.

Notes

This paper addresses a critical gap between laboratory demonstrations of accuracy nudges and real-world platform-scale feasibility. The two independent field experiments (Facebook industry-run; Twitter academic-run) provide rare causal evidence at scale, though several limitations bear note:

Strengths:

• Sample size: Among the largest field experiments on misinformation interventions (Facebook 33M users). Field setting eliminates artificiality of lab and survey contexts.
• Multiple platforms and implementations: Replication across Facebook (post-level exposure, Meta's fact-checking classifier) and Twitter (user-level targeting, domain-quality ratings) increases confidence in generalizability.
• Content-neutral approach: Avoids reliance on fact-checking infrastructure or centralized misinformation detection, making it compatible with encrypted platforms and privacy-preserving architectures—a genuine advantage over content-moderation approaches.
• Mechanism evidence: Post-experimental survey data and specificity tests (effects on misinformation, not general engagement) support predicted attention-redirection mechanism.

Limitations:

• Facebook 3-week result: No sustained effect over full campaign; suggests single or few exposures insufficient. Repeated messaging may require much higher ad frequency (authors note only 3.2 ads per user per 3 weeks, likely below saturation).
• Heterogeneity: Twitter effects vary significantly across experiments; investigators note effect driven primarily by highly active sharers. Unclear whether effects extend to lurkers, casual sharers, or broader populations.
• Measurement precision: Facebook relies on binary misinformation classifications (from fact-checkers or domain classifiers) rather than nuanced credibility measures; Twitter uses domain-quality thresholds (≤0.70 on 0–1 scale) which are somewhat arbitrary.
• Short time windows: Primary outcomes measured 60 minutes (Facebook) to 8–22 days (Twitter); longer-term behavioral persistence unknown. Effects may be temporary, requiring continuous intervention.
• External validity concerns: Ad delivery may not reflect organic exposure; users scrolling past ads without engagement may show zero effect, limiting ITT-to-ATT translation.

The paper makes a genuine contribution: it's the first large-scale field evidence that accuracy prompts work in uncontrolled platform environments. However, the practical significance remains modest (2.6% to 6.3% reductions relative to control). Whether sustained at scale with repeated ad campaigns, or at what cost, remains open.