Fake News Theories: Harnessing Disciplinary Insights for Computational Modeling, Detection, and Explanation

Zhaoyang Cao; Miriam Metzger; Reza Zafarani

In brief

This paper develops a theory-informed computational framework that translates cross-disciplinary social science theories into interpretable, measurable features for fake news detection. The authors systematically review theories from communication, psychology, and economics—organized into content-based, source-related, and network-level categories—and operationalize 11 theory-driven features (e.g., confidence heuristic, language vagueness, source credibility) using statistical methods and LLMs. Experiments on three benchmark datasets (RECOVERY, FAKEANDREALNEWS, GOSSIPCOP) show theory-derived features achieve 79.1% AUC and provide directional, interpretable signals, though individual feature improvements are modest; the framework prioritizes theoretical grounding and explainability over raw predictive performance, advancing human-centered approaches to misinformation detection.

Summary figure for Fake News Theories: Harnessing Disciplinary Insights for Computational Modeling, Detection, and Explanation

50Individual features achieve AUC 50–70.5% (F1: 47.9–67.5%)

79.1%Combined model on RECOVERY: AUC 79.1%, F1 76.8%

95.5%TF-IDF baseline: AUC 95.5%, F1 94.7% (higher due to full lexical coverage, but non-interpretable)

Contributions

Cross-disciplinary theoretical review: Systematic synthesis of theories from communication, psychology, information systems, and economics that explain why and how fake news persuades and spreads; organized into three categories (content, source, network) with explicit predictions for how fake news differs from real news.
Theory-to-computation operationalization: Translation of 11 theoretical concepts into measurable computational features using diverse technical approaches—lexicon-based measures (LIWC, VADER), statistical models (entity-grid coherence, NLI for logical consistency), and LLM-guided feature extraction (scarcity, illusory truth, information gap).
Empirical validation across datasets: Demonstrates theory-informed features capture meaningful signals for fake news classification with stable patterns across RECOVERY, FAKE_AND_REAL_NEWS, and GOSSIPCOP; shows source credibility as the dominant signal (when available) and reveals dataset-specific heterogeneity in feature importance.
Explainability and directionality: Uses SHAP analysis to reveal how individual features push predictions toward/away from misinformation label, providing transparent basis for model decisions; explicitly trades off predictive performance for interpretability and theoretical transparency.

Method

Theoretical Foundation

The authors identify theories in three categories:

I. Content-based theories predict differences in message composition between real and fake news:

Information Processing Theory and Confidence Heuristic: fake news uses more intense, sensational, or hyperbolic language
Emotion Theories and Negativity Bias: fake news contains more emotive and negative language
Expectancy Violation Heuristic: fake news has less clear, organized, accurate writing
Narrative Paradigm: fake news is less coherent with internal logical contradictions
Sourcing Theory: fake news includes less supporting evidence and external attribution
Illusory Truth Effect: fake news has more repetitive content
Linguistic Differences and Cognitive Load Theory: fake news has lower lexical/syntactic complexity, more personal pronouns and function words, fewer nouns and named entities
Information Manipulation Theory – Maxim of Manner: fake news contains more vague/ambiguous expressions
Uncertainty Reduction Theory: fake news headlines include more uncertainty-inducing cues (?, ..., !)

II. Source-related theories focus on credibility and trustworthiness:

Source Credibility Theory and Authority Heuristic: fake news comes from less reputable sources

III. Network-related theories (theoretically reviewed but not empirically evaluated due to dataset limitations):

Consensus Heuristic and Homophily Principle: fake news exhibits less external consistency across outlets, more linkages to known fake news sites

Feature Engineering

The authors operationalize 11 features:

Confidence Heuristic (binary): Classifies article as "certain" or "uncertain" using LIWC (Linguistic Inquiry and Word Count) by comparing frequency of certainty vs. uncertainty words.
Scarcity Principle (binary): LLM-guided feature using gemma-2-9b-it with prompt based on original scarcity paper definitions.
Negativity Bias (continuous): VADER sentiment analysis; compound score ranging [-1, +1], with scores below -0.5 indicating negative sentiment.
Coherence (continuous): Entity-grid model tracking syntactic role transitions (S/O/X) of entities across sentences; coherence computed as average log-probability of observed transitions.
Logical Consistency (continuous): Proportion of consecutive sentence pairs exhibiting ENTAILMENT relationship, using RoBERTa MNLI model.
Supporting Evidence (continuous): Aggregated heuristic combining five sub-features (URL frequency, quotations, temporal references, attribution phrases, proportion of URLs from .gov/.edu domains), normalized per thousand words.
Illusory Truth Repetition (continuous): GPT-4o-mini identifies distinct claims and counts repetitions; higher score indicates stronger repetitive persuasion.
Information Complexity (continuous): Flesch-Kincaid Reading Ease score; higher scores indicate greater complexity.
Language Vagueness (continuous): BERT-based classifier fine-tuned on vagueness dataset (privacy policies) to predict softmax probability of vagueness.
Information Gap (continuous): Combines punctuation-driven curiosity cues (?, !, ...) and topic surprisingness via LDA and KL divergence; fusion: IG(x) = w_P · P(x) + w_S · S(x).
Source Credibility (one-hot encoding): Publisher-level categorical encoding capturing latent source-specific credibility when metadata available.

Classification

Features fed to XGBoost for binary fake news classification. Experiments report AUC and macro-averaged F1 scores on benchmark datasets.

Results

Individual and Combined Performance

Individual features achieve AUC 50–70.5% (F1: 47.9–67.5%)
Combined model on RECOVERY: AUC 79.1%, F1 76.8%
TF-IDF baseline: AUC 95.5%, F1 94.7% (higher due to full lexical coverage, but non-interpretable)
Interpretation: Theory-informed model trades predictive performance for compactness and interpretability—deliberately sacrifices some predictive power for theoretical transparency.

Feature Importance (RECOVERY dataset)

Source Credibility: 45.8%
Information Complexity: 9.0%
Language Vagueness: 8.5%
Coherence: 7.2%
Illusory Truth: 6.5%
Logical Consistency: 6.4%
Negativity Bias: 5.8%
Information Gap: 5.4%
Supporting Evidence: 4.3%
Scarcity Principle: 0.9%
Confidence Heuristic: 0.3%

Key observation: Source credibility dominates when publisher metadata available (97.2% on GOSSIPCOP, 45.8% on RECOVERY), reflecting strong dataset-publisher associations; other content features show stable, moderate contributions.

Cross-Dataset Patterns

Source credibility consistent top feature across GOSSIPCOP and RECOVERY (where metadata available)
Content-based theories (information complexity, logical consistency, coherence, negativity bias) generalize moderately across datasets
Scarcity Principle more prominent in FAKE_AND_REAL_NEWS, suggesting stronger urgency rhetoric in that domain
Weak rank correlation between RECOVERY and FAKE_AND_REAL_NEWS (Spearman ρ=0.167), indicating dataset-specific heterogeneity

Content Analysis

Wilcoxon rank-sum tests show statistically significant differences (p < 0.001):

Fake news exhibits lower sentiment (stronger negativity bias)
Fake news shows lower information complexity (simpler narratives, reduced lexical/syntactic diversity)
Fake news displays greater linguistic vagueness (ambiguous/imprecise expressions)
Fake news demonstrates larger information gap (more uncertainty-inducing headline cues)

Feature Interactions

Pairwise combinations: Best two-feature pairs—confidence heuristic + supporting evidence (AUC 0.740), scarcity principle + supporting evidence (0.737)—suggest assertive language + verifiable evidence jointly capture persuasive intent.
Triple combinations: Best three-feature set—(confidence heuristic, scarcity principle, supporting evidence)—achieves AUC 0.745, reflecting interaction among language intensity, urgency, and evidence transparency. Gains are modest, not uniformly additive.

SHAP Analysis

Source credibility most decisive factor for distinguishing fake/real
Information complexity and negativity bias show consistent influence with directional effects: higher values (red) push toward misinformation; lower values (blue) shift opposite direction
Information gap and coherence demonstrate noticeable but less dominant effects
SHAP vs. feature importance: SHAP reveals directionality of feature impacts, whereas simple feature importance does not.

Feature Independence

Most pairwise correlations fall below |0.1|, indicating features capture unique information
Feature correlation heatmap validates diversity of theoretically-grounded features

Notes

Strengths:

Systematic, disciplinary cross-review grounding computational features in established theory; rare and valuable for interpretability.
Comprehensive operationalization: diverse techniques (lexicon, statistical, LLM) show how different theoretical concepts can be implemented.
Multi-dataset evaluation demonstrates generalization of some features (information complexity, coherence) while surfacing dataset-specific heterogeneity.
SHAP analysis provides directional transparency often missing from black-box fake news detectors; useful for practitioners.
Honest about trade-off between interpretability and predictive performance, rather than overstating predictive gains.

Weaknesses:

Source credibility dominance on some datasets likely reflects dataset artifact (publishers strongly associated with labels) rather than intrinsic strength of source credibility signal in the wild.
Network-level theories (consensus heuristic, homophily) theoretically reviewed but not empirically evaluated; benchmark datasets lack propagation traces and inter-source network metadata.
Weak cross-dataset feature rank correlations (ρ=0.167) suggest limited universality of feature importance hierarchy; relative contribution of theories is dataset-dependent, limiting generalization.
Some feature extractions rely on pretrained/LLM models (BERT for vagueness, LLMs for claim identification) sensitive to domain shift and prompting; robustness across diverse news domains not systematically evaluated.
Comparison to TF-IDF baseline shows substantial predictive cost (~16% AUC gap); unclear whether this trade-off is acceptable for specific application scenarios (e.g., real-time detection vs. ex-post auditing).

Open questions:

How do theory-driven features perform on out-of-domain news corpora (non-English, emerging fake news narratives, new platforms)?
Can network-level theories be operationalized and evaluated on datasets with propagation or inter-source linking metadata?
What is the relative importance of individual theories vs. their combinations for different fake news genres (political, health, clickbait, satire)?
How sensitive are LLM-based features to model choice, prompting strategy, and instruction tuning? Can smaller/open-source LLMs replicate performance?

Significance for the field: This work exemplifies a needed shift toward theory-informed, explainable fake news detection. Rather than pursuing ever-higher black-box accuracy on benchmark datasets, it demonstrates how social science theory can structure interpretable detection systems, making predictions auditable and trustworthy for domain experts (journalists, fact-checkers, policy researchers). The systematic mapping between theory and computation is a methodological contribution applicable beyond misinformation to other credibility-assessment tasks.