Fact-checking and corrections

Fact-checking refers to efforts to investigate claims already in the news or on social media (Graves, 2016) and to provide corrections or rebuttals. Examples include dedicated fact-checking organizations (Snopes, FactCheck.org, PolitiFact), fact-checking departments within news outlets, and myth-busting efforts by health/scientific institutions (CDC, WHO).

Despite its intuitive appeal—fighting falsehoods with facts—research shows fact-checking has limited effectiveness and can sometimes backfire. Psychological research identifies several barriers: (1) motivated reasoning, where audiences defend pre-existing beliefs against corrections; (2) backfire effects, where corrections reinforce false beliefs in some contexts; (3) the "continued influence effect," where people continue to rely on debunked information even after correction; (4) emotional resonance, where emotionally-charged false claims outcompete flat factual rebuttals in persuasiveness.

Risk communication research adds: fact-checking is itself a form of risk communication, and its effectiveness depends on trust in fact-checkers, agreement on what "the risk" actually is, and transparent handling of uncertainty. When audiences define misinformation risk differently from fact-checkers (e.g., audiences blame journalists; fact-checkers are affiliated with journalists), fact-checkers' credibility suffers.

Key papers

• Wang & Shu (2023) — Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models — FOLK method using FOL decomposition and knowledge-grounding to guide LLMs; achieves state-of-the-art on HoVER, FEVEROUS, and SciFactOpen while generating high-quality explanations
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection — Teaches language models to self-critique using reflection tokens; improves factuality and citation accuracy on fact verification tasks
• Atlas: Few-shot Learning with Retrieval Augmented Language Models — Jointly trained retrieval-augmented model achieving state-of-the-art few-shot fact-checking (56.2% with 15 examples, 80.1% full-dataset on FEVER) with compact parameters
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Proposes RAG for combining retrieval with generation, including evaluation on FEVER fact verification task
• Rumor Cascades — Analyzes effect of Snopes fact-check comments on rumor cascade deletion rates and propagation
• Augenstein et al. (2019) — MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims — largest publicly available real-world dataset (34,918 claims from 26 fact-checking websites) with evidence pages and entity linking; multi-task learning approach jointly ranks evidence and predicts veracity across domains with heterogeneous label schemas
• Kim et al. (2017) — Leveraging the Crowd to Detect and Reduce the Spread of Fake News and Misinformation — uses stochastic optimal control to schedule crowd-sourced fact-checking; the CURB algorithm decides which stories to send for fact-checking and when, minimizing the spread of misinformation given limited fact-checking resources.
• Shiralkar et al. (2017) — Finding Streams in Knowledge Graphs to Support Fact Checking — Unsupervised network-flow approach (Knowledge Stream) treating knowledge graphs as flow networks; computes truth scores via relational similarity and edge capacities; achieves performance comparable to supervised baselines with high interpretability
• Shao et al. (2018) — Anatomy of an online misinformation network — network analysis of fact-checking vs. misinformation spread on Twitter during 2016 US election; finds fact-checking nearly disappears in network core and is ineffective at competing with low-credibility claims
• Hanselowski et al. (2019) — A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking — SNOPES corpus with 6,422 validated claims and 14,296 documents from heterogeneous web sources (news, blogs, social media); comprehensive annotations for document retrieval, evidence extraction (fine-grained), stance detection, and claim validation; error analysis identifies challenges of unreliable sources
• Zlatkova et al. (2019) — Fact-Checking Meets Fauxtography: Verifying Claims About Images — dataset of 1,233 image-claim pairs (838 from Snopes, 395 from Reuters Pictures of the Year) labeled as true/false; uses reverse image search to extract features from web pages returning the image, combined with claim text and image metadata; linear SVM achieves 80.1% accuracy, substantially above baseline; identifies URL domains and media source credibility as most informative features
• Thorne et al. (2018) — FEVER: A Large-Scale Dataset for Fact Extraction and VERification — dataset of 185,445 human-verified claims with Wikipedia evidence; three-class labels (SUPPORTED/REFUTED/NOT ENOUGH INFO) with sentence-level evidence annotation; baseline system combines document retrieval, sentence selection, and textual entailment achieving 31.87% accuracy with correct evidence
• Thorne et al. (2018) — The Fact Extraction and VERification (FEVER) Shared Task — shared task benchmark for automatic fact verification combining evidence retrieval from Wikipedia with claim classification; dataset of 185,445 claims, best system achieves 64.21% score
• Rashkin et al. (2017) — Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking — linguistic analysis of PolitiFact data with 6-point graded truthfulness scale; demonstrates that stylistic features (hedging, subjectivity, intensifiers) help predict statement veracity
• Lazer et al. (2018) — The Science of Fake News — comprehensive review documenting why fact-checking has limited effectiveness due to confirmation bias, selective exposure, and motivated reasoning; discusses when corrections can backfire
• Hameleers et al. (2020) — A Picture Paints a Thousand Lies? — experimental evidence that fact-checkers effectively counter multimodal disinformation (text+image Twitter posts) despite visual content's credibility advantage; fact-checker modality (visual vs. text) has minimal effect, but motivated reasoning moderates effectiveness—fact-checkers are most persuasive when reaching people already skeptical of the false claim.
• Nieminen & Rapeli (2019) — Fighting Misperceptions and Doubting Journalists' Objectivity: A Review of Fact-checking Literature — comprehensive literature review of 48 studies on political fact-checking, organized by three research areas: (1) effectiveness in reducing misperceptions (mixed results, including backfire effects); (2) fact-checking as a profession (methodological inconsistencies and reliability concerns); (3) public opinion about fact-checking. Identifies geographic and institutional bias in the literature (88% US-focused).
• Walter et al. (2020) — Fact-Checking: A Meta-Analysis of What Works and for Whom — comprehensive meta-analysis of 30 studies quantifying fact-checking effectiveness (d = 0.29) and identifying key moderators: motivated reasoning, political ideology, message design, and context; finds pro-attitudinal corrections much more effective than counter-attitudinal ones, visual elements often backfire, and campaign messaging is harder to correct.
• Graves (2016) — Mapping the institutional roots of the global fact-checking movement — ethnographic mapping of fact-checking organizations globally across journalism, academia, and politics/civil society axes; documents institutional diversity and contested professional boundaries across countries.
• Krause et al. (2020) — Fact-checking as risk communication — argues that fact-checking fails as a strategy without trust, and that competing definitions of "misinformation risk" undermine fact-checker credibility in polarized environments.
• Lewandowsky et al. (2012) — Misinformation and its correction — foundational psychology review of why corrections fail and how to design effective debiasing; identifies cognitive mechanisms (mental models, source confusion) and evidence-based fixes (warnings, alternative explanations, repeated corrections).
• Pennycook et al. (2020) — Accuracy-nudge intervention — shows that a simple, content-neutral nudge (prompting people to rate accuracy) nearly triples truth discernment in sharing decisions, suggesting effectiveness depends less on the correction content and more on making accuracy salient.
• Vo & Lee (2020) — Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News — novel framework to retrieve fact-checking articles that address claims in original tweets; uses multimodal attention network combining textual and visual matching signals; demonstrates that multimodal retrieval substantially outperforms text-only baselines on Snopes and PolitiFact, enabling proactive warning systems that inform users of fact-checked misinformation.
• Lee et al. (2020) — Misinformation Has High Perplexity — proposes using language model perplexity as a signal for falseness when the model is trained on truthful evidence; achieves 75% accuracy on scientific COVID-19 claims using GPT-2 with minimal labeled data; releases Covid19-scientific and Covid19-politifact test sets.
• TELLER: A Trustworthy Framework For Explainable, Generalizable and Controllable Fake News Detection — decomposes fact-checking into interpretable question templates answered by LLMs; decision system learns logic rules for claim verification; demonstrates how automated systems can achieve explainability, generalizability, and human controllability in verification tasks

Limitations of fact-checking

• Low general efficacy: reviews show fact-checks correct beliefs in some people but fail or backfire in others, particularly when beliefs are value-laden or identity-protective.
• Slow at scale: journalists and fact-checkers cannot keep pace with viral misinformation; by the time a fact-check is published, thousands may have seen the false claim.
• Credibility barriers: 48% of Americans believe fact-checkers favor one side (Pew, 2019b); when affiliated with traditional media, they inherit low press trust.
• Trust-dependent: effectiveness depends on audience trust in the fact-checker—a prerequisite that is often violated in polarized environments.
• Emotion vs. objectivity: fact-checks tend to be emotionally flat; false claims often carry emotional resonance that factual rebuttals cannot match.

Promising approaches

• Pre-bunking ("inoculation"): exposing audiences to weakened arguments before encountering strong misinformation (van der Linden et al., 2017; Cook et al., 2017).
• Accuracy nudges: reminding people to consider accuracy before sharing (Pennycook et al., 2020).
• Value-congruent framing: connecting corrections to audience values rather than relying on objectivity (Kunda, 1990; Ho et al., 2011).
• Trusted institutional partnerships: fact-checking via highly-trusted sources (CDC, WHO) rather than media-affiliated organizations.
• Transparent uncertainty: acknowledging what is genuinely unknown, which does not reduce trust (Van Der Bles et al., 2020).

Connections

• Risk communication — fact-checking is a form of risk communication; must address trust, risk-definition differences, and uncertainty.
• COVID-19 misinformation and the infodemic — dominated by fact-checking efforts; Krause et al. (2020) argues these fail without addressing psychological and institutional barriers.
• Trust in institutions and communicators — central to fact-checking effectiveness.
• Misinformation Interventions — fact-checking is one intervention type; others include pre-bunking, nudges, source credibility labeling.