Long-Term Effectiveness of Inoculation Against Misinformation: Three Longitudinal Experiments

Contributions

• Replicates the short-term effectiveness of the Bad News game intervention in a pre-registered study with a larger and more diverse test battery than the original Roozenbeek & van der Linden (2019) work
• Systematically measures the longevity of inoculation effects across three time-points spanning weeks to months
• Demonstrates that active psychological inoculation effects decay over time without reinforcement, consistent with broader inoculation theory predictions
• Tests whether long-term effects are driven by item-response memorization or broader transfer of critical thinking skills (Experiment 3)
• Provides empirical evidence for the "booster shot" concept in psychological inoculation—regular re-exposure maintains resistance to persuasion

Method

All three experiments used a randomized pretest-posttest design. Participants (recruited via Prolific) rated the reliability of news headlines (7-point Likert scale) before and after playing either the Bad News game (inoculation condition) or Tetris (control condition). Headlines included both factual news items and fake news items employing specific misinformation techniques (discrediting, emotional appeals, polarization, impersonation, conspiracy theories, trolling).

Experiment 1 (N=151): Measured inoculation effects at pretest (T1), immediate posttest (T2), one week later (T3), one month later (T4), and three months later (T5). This design allowed continuous assessment to investigate whether regular testing itself maintains the effect.

Experiment 2 (N=194): Replicated Experiment 1's design but without the intermediate T3 and T4 assessments. Participants were tested at pretest (T1), immediate posttest (T2), and two months later (T3). This design examined whether decay occurs when participants are not regularly re-exposed to the measurement task.

Experiment 3 (N=170): Replicated Experiment 1 design (up to T3) but used different news items at each assessment than in the training game itself, and used a balanced 6:6 fake-to-real item ratio instead of 18:3. This addressed the alternative explanation that long-term effects reflect item-specific memorization rather than transfer of broader critical thinking.

Results

Immediate inoculation effect: Participants in the Bad News condition rated fake news as significantly less reliable than the control group (F(1,114)=29.86, p<.001). The effect was robust and immediate.

Experiment 1 (with regular assessment): The inoculation effect remained statistically significant at three months (T5), demonstrating that with repeated measurement, effects are durable. However, effect size showed some decline over time compared to the immediate posttest.

Experiment 2 (without intermediate measurement): Without regular "booster" exposure via reassessment, the effect showed significant decay. By two months (T3), the inoculation effect was no longer statistically significant, indicating that the longevity observed in Experiment 1 was partly sustained by the measurement process itself.

Experiment 3 (transfer and memorization): The inoculation effect generalized to different items and remained significant one week later, suggesting that effects are not purely due to memorization of specific training items but reflect broader learning of how to identify misinformation techniques.

Notes

This is one of the first systematic investigations of inoculation decay in the fake-news context. A key insight is that the measurement process itself appears to serve as a booster—regular testing maintains effects that otherwise decay. This has practical implications: deploying inoculation in the real world may require periodic reinforcement, not just one-time exposure.

The distinction between Experiment 2 (no booster = decay) and Experiment 1 (with regular assessment = stability) suggests two mechanisms: (1) passive decay of memory/skills over time, and (2) active reinforcement through practice or measurement. The study does not disambiguate these fully but raises the question of what constitutes an effective booster in practice.

Experiment 3's finding that effects transfer to novel items with different fake-to-real ratios strengthens the claim that the intervention teaches generalizable critical thinking about misinformation techniques, not item-specific memorization.