Measuring Belief in Fake News: The Truth About Vulnerability to Fake News Online

Key claims and findings

1. Ordinary people are not good at identifying fake news. When shown recently published articles and asked whether they are true, false, or misleading, untrained individuals perform poorly at distinguishing false news from true news.

2. Political identity predicts susceptibility more than demographics. The study found that politics—partisan identity, ideological alignment—is the strongest predictor of who believes fake news, not conventional demographic variables like education or age.

3. Searching for corroborating evidence backfires. Encouraging people to search the internet for evidence before deciding whether an article is false makes them worse at identifying false stories. Respondents who searched were more likely to believe false articles were true.

4. The internet is full of junk. The reason searching backfires is that Google searches for the false articles being tested returned corroborating false or misleading information, leading people to conclude the false article must be true.

5. This effect is conditional. The backfiring effect of searching only occurs when other corroborating false information actually exists online. When no such information exists, searching doesn't harm performance.

6. COVID-19 findings mirror political misinformation patterns. When the researchers ran a follow-up study in May–June 2020 focused on COVID-19 misinformation, the patterns of belief and the individual-level predictors largely held across the health-information domain.

7. Crowdsourced fact-checking fails with simple rules. Simple aggregation methods (taking the mode of the crowd, or random sampling of crowd judgments) performed very poorly at identifying false news, suggesting naive crowdsourced fact-checking approaches will not work.

8. Machine learning shows promise. More complex machine learning methods that aggregate crowd judgments and incorporate multiple signals showed potential for improving fake news detection, pointing toward hybrid human-AI fact-checking systems.

9. Incentives matter, but weakly. Paying people small amounts for correct answers improved performance, but the effect was smaller than the political-identity effect, suggesting the problem is not purely rational uninformededness but partly motivated reasoning.

Research design

The study used a novel methodology:

• Pre-registered article selection: Five news sources (left-of-center mainstream, right-of-center mainstream, right-leaning false news sites, left-leaning false news sites, and miscellaneous sites) were monitored daily. The most popular article from each source was automatically selected without researcher discretion, eliminating selection bias.
• Parallel professional and crowd assessment: Each article was sent to professional fact-checkers (5–6 per study) to establish ground truth, and to ~90 crowdworkers per article to measure lay accuracy.
• Three randomized experiments: Embedded within the main study were three 2-week experiments testing whether searching for corroborating evidence, financial incentives, or varying headline-only vs. full-article presentation improved accuracy.
• Article volume: Study 1 (Nov 2019–Feb 2020) included 155 articles (37 false or misleading). Study 2 (COVID-19, May–June 2020) included 40 articles (13 false or misleading).

Context and institutional affiliation

The work emerges from NYU's Center for Social Media and Politics, co-directed by Tucker, in collaboration with Nate Persily at Stanford's Cyber Policy Center. Tucker frames the research within the emerging field of social media and politics, a subfield focused on how digital platforms shape political behavior, polarization, and information integrity. The lecture series at UC Berkeley was part of a broader initiative called "Solidarity and Strife: Democracies in a Time of Pandemic," funded by the Social Science Research Council.

Notes

This lecture is valuable for several reasons. First, it presents large-scale empirical evidence on a fundamental question: how often do ordinary people actually believe false news in real time? Many studies infer vulnerability from surveys or lab experiments; this study uses actively published, contemporaneous articles. Second, the pre-registered methodology and professional fact-checker ground truth offer higher rigor than typical crowdwork-based studies. Third, the counterintuitive finding that searching for evidence backfires directly challenges a widespread digital-literacy intervention (encouraging independent verification), suggesting interventions must be designed with care. Finally, the comparison of political misinformation and COVID-19 misinformation in the same framework during the early pandemic is historically valuable: the patterns largely held despite the different stakes and content domains.

Tucker is explicit about the limitations of crowdsourced fact-checking under naive aggregation rules, yet concludes optimistically that machine-learning-assisted approaches and hybrid human-AI systems may offer a path forward—a nuanced middle ground between dismissing both crowds and automation.