Liar Dataset¶
The LIAR dataset is one of the largest and most widely-used benchmarks for fake news detection research. It contains 12,836 short political statements from PolitiFact, labeled with six credibility classes and rich metadata about speakers and statements.
Dataset characteristics¶
Size and scope: 12,836 news items collected from PolitiFact public fact-checking data (2007–2016). Focuses on political statements in the US political context.
Fine-grained labels: Six-way classification reflecting a spectrum of truthfulness: - Pants-fire (most false) - False - Barely-true - Half-true - Mostly-true - True (most true)
This ordinal structure is more realistic than binary fake/real classification and better reflects actual credibility assessment.
Rich metadata: Each statement includes: - Speaker name, party affiliation, job title - Speaker historical credibility context (credit history of past statements) - Statement context (interview, debate, speech, campaign ad, etc.) - Subject matter (education, economy, health care, etc.) - Statement text
This metadata enables detection methods to exploit speaker profiles and historical patterns, not just textual content.
Usage in research¶
LIAR has become a standard benchmark for evaluating fake news detection methods. Studies often compare approaches across different labeled-data regimes (10%, 20%, 30% labeled with semi-supervised learning) to assess robustness in realistic, label-scarce scenarios.
Many graph-based methods have adapted LIAR as a heterogeneous graph problem, constructing edges between news items based on shared speakers, topics, or content similarity.
Key papers using LIAR¶
- A Decision-Based Heterogeneous Graph Attention Network for Multi-Class Fake — DHGAT: Multi-class detection with speaker-profile-derived heterogeneous graphs; achieves ~4% accuracy improvement in semi-supervised settings.
- Neighborhood-Order Learning Graph Attention Network for Fake News Detection — NOL-GAT: Adaptive neighborhood order learning; evaluated across five datasets with LIAR as a baseline.
- Wang (2017) — Liar, Liar Pants on Fire: A New Benchmark Dataset for Fake News Detection: Original paper introducing the dataset.