Towards Fine-Grained Reasoning for Fake News Detection¶
Authors: Yiqiao Jin, Xiting Wang, Ruichao Yang, Yizhou Sun, Wei Wang, Hao Liao, Xing Xie Venue: ACL, 2021 — arXiv:2110.15064
TL;DR¶
The paper proposes FinerFact, a framework for fake news detection that models reasoning at the claim-evidence level rather than the document level. It constructs claim-evidence graphs and uses mutual-reinforcement-based ranking to identify salient evidence, then applies a bi-channel kernel graph network to reason over evidence groups. The method achieves 91.7% F1 on PolitiFact and 86.4% F1 on GossipCop, outperforming prior work while providing human-interpretable explanations of why articles are classified as true or false.
Contributions¶
- Fine-grained reasoning framework: Models fake news detection as reasoning over claims and evidence rather than coarse-grained article classification, following human information-processing principles.
- Mutual-reinforcement-based evidence ranking: Ranks evidence by integrating human knowledge about attribute salience (e.g., which sources are credible, which evidence types are important) and topological salience.
- Bi-channel kernel graph attention network: Extends KGAT to incorporate both textual and social channels, enabling the model to reason about how different types of evidence support or contradict claims.
- Interpretability: The framework provides explanations by identifying which evidence groups are most important and which tokens within evidence are most salient for each prediction.
Method¶
Claim-Evidence Graph Construction¶
The method first extracts claims from news articles and associates them with supporting evidence. To identify important claims, it uses topic-aware claim-evidence association: the paper models major viewpoints from evidence (via LDA) and selects claims that align with the most salient topics. For each selected claim, it gathers evidence from posts, users, and keywords mentioned in the supporting context.
The framework builds a mutual reinforcement graph \(M = \{A_{xy}\}\) where nodes represent posts, users, and keywords and edges encode relationships: - Post-post edges: cosine similarity based on BERT embeddings - User-post edges: retweet/reply relationships - Keyword-post and keyword-keyword edges: co-occurrence in posts
The saliency of evidence is computed iteratively: a piece of evidence is salient if it appears often, is shared by important users, or appears in high-engagement posts. This is formalized as attribute salience \(e_u\) for each user, which balances follower count, friend count, and verification status.
Graph-Based Fine-Grained Reasoning¶
For each claim, the method constructs a claim-evidence graph \(G\) where nodes are evidence groups (posts, users, keywords) and edges represent co-occurrence. The model then reasons over this graph using a Prior-Aware Bi-Channel KGAT:
- Textual channel: Encodes claim tokens and evidence text via BERT, then uses kernel matching to compute fine-grained attention between claim-token pairs and evidence tokens.
- Social channel: Uses user-level features (follower count, account age, verification status) and post-level features (engagement, timestamp) to weight evidence.
The node-level prediction aggregates signals from both channels via a two-step process: - First, compute per-channel importance scores for each evidence piece - Then, aggregate across channels and normalize to obtain final node importance
The final article-level prediction combines predictions from multiple evidence groups (topics), with the model learning to weight evidence groups differently.
Results¶
Evaluated on PolitiFact (815 true, 372 fake articles) and GossipCop (4219 true, 3393 fake articles):
| Dataset | Model | Precision | Recall | F1 | Accuracy | AUC |
|---|---|---|---|---|---|---|
| PolitiFact | Baseline KAN | 0.8687 | 0.8499 | 0.8539 | 0.8556 | 0.9197 |
| PolitiFact | FinerFact | 0.9196 | 0.9047 | 0.9172 | 0.9172 | 0.9384 |
| GossipCop | Baseline KAN | 0.7764 | 0.7696 | 0.7713 | 0.7766 | 0.8435 |
| GossipCop | FinerFact | 0.8615 | 0.8579 | 0.8635 | 0.8620 | 0.8637 |
FinerFact outperforms strong baselines by 6.3% F1 on PolitiFact and 5.5% F1 on GossipCop. Ablation studies show that both the mutual-reinforcement evidence ranking and the bi-channel kernel architecture contribute substantially to performance.
Connections¶
- Related to Evidence-based fake news detection via the focus on evidence selection and ranking.
- Related to Claim Verification as it models claim-level verification rather than full-article verification.
- Related to Graph Neural Networks via the use of KGAT and graph-based reasoning.
- Related to Explainable fake news detection by providing interpretable predictions through evidence importance and token-level attention.
- Related to Computational fact checking for the connection to fact-checking as a reasoning process.
Notes¶
The paper makes a compelling case for fine-grained reasoning by showing that evidence quality and type matter. The human-interpretability angle is valuable: the model successfully identifies that articles claiming "Hillary will be blamed for her hacking" rely on suspicious evidence (e.g., anonymous server hack claims), while articles about "Trump covering up DNC server hacks" correctly prioritize more mainstream coverage. The limitation around evaluation of interpretability (no ground-truth annotations for explanation quality) is acknowledged and suggests follow-up work combining the approach with human-in-the-loop refinement.