Fake news game confers psychological resistance against online misinformation

Authors: Jon Roozenbeek, Sander van der Linden

Venue: Palgrave Communications, Vol. 5, Article 65, 2019 — DOI

TL;DR

This large-scale empirical study demonstrates that a free ~15-minute online browser game teaching six common misinformation production techniques (polarisation, emotional manipulation, conspiracy theories, trolling, discrediting opponents, impersonation) significantly reduces perceived reliability of misinformation across N=15,000 participants. The intervention works through active inoculation theory — preemptively exposing people to weakened examples of deceptive techniques builds cognitive immunity — and is effective regardless of education, age, political ideology, or cognitive style.

Contributions

• Operationalization of active inoculation theory in a gamified, interactive learning environment rather than passive reading of refutations; shows that experiential learning activates deeper cognitive engagement
• Broad-spectrum inoculation approach targeting six general techniques underlying diverse misinformation rather than content-specific vaccines (e.g., single conspiracy theories); demonstrates cross-domain spillover effects
• Large-scale empirical validation (N=15,000 pre-post paired responses) showing robust main effect: participants rated fake news significantly less reliable post-gameplay (average Mdiff = −0.55, Cohen's d = 0.52)
• Empirical evidence for equal-opportunity intervention — effect robust across political ideology (liberals and conservatives both improve ~0.55–0.59 points), age groups, education levels, and cognitive reflection style; largest effect for those most susceptible to misinformation at baseline (d = 0.89)
• Methodological innovation: uses fictional but realistic misinformation examples to avoid memory confounds (participants do not know the stories to rule them out, they must recognize the deceptive technique)

Method

The Bad News game is an interactive choice-based adventure (~15 minutes) where players assume the role of a fake news creator trying to maximize followers and credibility. Players encounter six scenarios, each teaching one deception technique:

1. Impersonation: Creating fake accounts mimicking real people/organizations (e.g., fake HBO Twitter account)
2. Emotional content: Producing material triggering fear, anger, or empathy
3. Group polarisation: Amplifying existing grievances between social groups
4. Conspiracy theories: Creating or amplifying alternative explanations for events assuming hidden elites control them
5. Discrediting opponents: Attacking or delegitimising sources of criticism via ad hominem or denial
6. Trolling: Deliberately inciting reactions through bait and provocative content

Players are rewarded (followers/credibility) for choosing deceptive options and punished for ethical journalistic choices. At 15 minutes in-game, players encounter a voluntary pre-post survey. Researchers collected N=43,687 survey responses over three months (Feb–April 2018), yielding n=14,266 paired pre-post responses (exact n varies by item: 14,163–14,266).

Measures: Participants rated reliability (1=unreliable, 7=reliable) of six headlines/tweets both before and after gameplay: - Two control questions: real news items (Trump wall announcement, Brexit timing) to detect social desirability effects - Four treatment questions: tweets/headlines embedding impersonation, conspiracy, discrediting, or polarisation

Sampling: Convenience sample via university press release covered by BBC; self-selected, opt-in; overrepresents males (75%), higher education (47%), younger players (18–29, 47%), and left-liberal ideology (59%)

Analysis: One-way repeated measures MANOVA with univariate follow-ups using conservative Bonferroni correction. Authors emphasize effect sizes (Cohen's d, Hedges' g) alongside p-values given large sample size.

Results

Main effects: All treatment items showed significant decrease in perceived reliability post-gameplay: - Impersonation: Mpre=3.00, Mpost=2.30, d=0.36, Hedges g=0.33, p<0.0001 - Conspiracy: Mpre=2.47, Mpost=1.97, d=0.35, Hedges g=0.32, p<0.0001 - Discrediting: Mpre=2.37, Mpost=1.92, d=0.30, Hedges g=0.26, p<0.0001 - Polarisation (post-hoc sample, n=885 paired): Mpre=2.57, Mpost=2.30, d=0.16, Hedges g=0.15, p<0.0001

Control items showed negligible change (Trump: d=0.03; Brexit: d=0.04), demonstrating that players did not simply become skeptical of all media but selectively learned to detect specific deception tactics.

Subgroup analyses: - Political ideology: Both liberals and conservatives improved similarly (liberals: Δ=−0.55; conservatives: Δ=−0.59; no significant difference, p=0.08); same pattern for polarisation-specific items - Age: Small difference: younger players improved slightly more (d=0.04, negligible), likely floor effect - Education & cognitive reflection: No significant differences across education level or cognitive reflection test performance - Gender: Small effect favoring females (d=0.05, negligible) - Prior susceptibility (median split on pre-test scores): Largest effect among most vulnerable participants (Δ=−1.06 vs. Δ=−0.19 for less susceptible; d=0.89)

Overall average inoculation effect across all badges: Mpre=2.61, Mpost=2.06, d=0.52, Hedges g=0.43.

Connections

• Directly applies inoculation theory to misinformation via active (experiential) rather than passive learning
• Grounded in psychological drivers of belief — builds cognitive resistance through recognition of manipulation techniques
• Related to cognitive reasoning and reflection though effect persists independent of cognitive style
• Complements debunking approaches by shifting to pre-emptive (rather than reactive) intervention
• Shares broad-spectrum intervention philosophy with Ecker et al. (2022) on designing multi-domain resistance
• Empirically tests psychological theories articulated in Ecker et al. regarding prebunking effectiveness and barrier removal
• Relevant to social media and misinformation in designing scalable, in-situ interventions accessible to millions of users

Notes

Strengths: - Novel experimental design: experiential (game-based) inoculation contrasts with traditional passive inoculation literature; players actively reason about deceptive strategies rather than passively reading refutations - Large, real-world sample (N=15K) provides statistical power and ecological validity; not confined to lab setting or student population - Fictional but realistic content avoids memory confounds — participants recognize deception technique rather than simply "knowing" a story is false - Honest treatment of limitations: acknowledges lack of randomized control group (ethical constraint: free, open-access intervention), self-selected sample bias, inability to measure confidence in participants' reliability ratings - Effect sizes consistent with broader inoculation literature (~d=0.3–0.5 is meaningful in persuasion research) and contextualized against real-world examples (Brexit decided on 4% margin) - Robustness across subgroups strengthens claim for "broad-spectrum" vaccine without psychological reactance

Weaknesses: - No randomized control group, only pre-post within-subjects; while real-news control items mitigate demand-effect concerns, alternative explanations not entirely ruled out - Sample heavily skewed toward males, higher education, younger, and liberal-leaning; generalizability to older, less-educated, or right-leaning populations unclear - No long-term follow-up; unclear whether inoculation effects persist weeks/months later or if "booster shots" (replaying) are needed - Doesn't address negative side effect concern (could game teach malicious actors how to spread misinformation more effectively?); authors argue risk low because political/financial motivation absent and techniques already public, but empirical test absent - Limited mechanistic insight into which design features (player agency, game narrative, reward structure) drive the effect; ablation studies comparing game to simpler inoculation formats missing - Cannot determine whether effect stems from learning about techniques, increased skepticism generally, or some other factor

Follow-up opportunities: - Randomized controlled trials comparing Bad News game to passive inoculation and traditional media literacy curricula across cultures - Longitudinal tracking of inoculation decay and optimal booster-shot intervals - Mechanistic studies isolating key game elements (choice-based structure, avatar identification, reward feedback) driving learning - Investigation of potential iatrogenic effects among actors with different motivations (e.g., do sophisticated manipulators learn refinements?) - Deployment in school curricula and assessment of classroom uptake, peer effects, and parental engagement - Cross-domain generalization: does learning misinformation techniques transfer to recognizing other information disorders (conspiracy theories, health misinformation)?