Skip to content
A Decision-Based Heterogeneous Graph Attention Network for Multi-Class Fake News Detection

A Decision-Based Heterogeneous Graph Attention Network for Multi-Class Fake News Detection

Authors: Batool Lakzaei, Mostafa Haghir Chehreghani, Alireza Bagheri Affiliation: Amirkabir University of Technology (Tehran Polytechnic), Iran Preprint: arXiv 2501.03290, January 2025

TL;DR

Proposes DHGAT (Decision-based Heterogeneous Graph Attention Network), a semi-supervised method for six-class fake news classification on the LIAR dataset. Unlike standard GNNs that apply uniform neighborhood aggregation, DHGAT models news as heterogeneous graphs with multiple edge types derived from speaker profiles and enables each node to dynamically select optimal neighborhood types per layer via a decision network. Achieves ~4% accuracy improvement over baselines and demonstrates robustness with 10–30% labeled data.

Contributions

  • Introduces heterogeneous graph representation of the LIAR dataset, using speaker metadata (name, party affiliation, job title, state, context, credit history) to define diverse edge types between news items.
  • Proposes a two-network architecture combining a decision network (to select optimal neighborhood types) and a representation network (to update embeddings based on selected neighborhoods).
  • Enables per-node per-layer dynamic neighborhood selection via Gumbel-Softmax sampling, allowing each node to choose which edge types to aggregate in each layer independently.
  • Addresses limitations of homogeneous GNN approaches (fixed neighborhood type, inability to leverage diverse contextual information) and prior heterogeneous methods that require global weight matrices.
  • Demonstrates multi-class (six-class) classification on LIAR with semi-supervised learning, outperforming baselines in both labeled-data-rich and label-scarce regimes.

Method

The DHGAT pipeline consists of three components:

  1. Heterogeneous graph construction: News items are nodes; edges are drawn from speaker profile features. The LIAR dataset yields nine edge types: speaker, context, subject, party-affiliation, job-title, state, and three K-nearest-neighbor similarity edges (KNN-5, KNN-6, KNN-7). Different subsets of these edge types define different graph structure options.

  2. Decision-based Heterogeneous Graph Attention Network:

  3. Decision network (Φ): A GATv2 network that outputs a probability distribution over possible neighborhood types for each node in each layer. Uses Gumbel-Softmax to sample a discrete neighborhood type γᵢ ∈ Γ where Γ is the set of all possible edge-type combinations (up to 2^|R| choices if |R| edge types are available).
  4. Representation network (Ψ): A second GATv2 network that updates node embeddings using only the selected neighborhood type, aggregating information from neighbors connected via edges of type γᵢ.

  5. The two networks are trained jointly: Φ learns which neighborhood type is optimal, while Ψ learns to extract and propagate information through that chosen neighborhood.

  6. Multi-class classification: Final embeddings from the last layer are fed to a Multi-Layer Perceptron with a custom loss function that combines cross-entropy loss (for labeled samples) with semantic distance loss (to minimize distance between predicted and true labels, accounting for label orderings like "false" < "barely-true" < "true").

Training uses the Gumbel-Softmax estimator to provide a differentiable approximation of discrete sampling, enabling backpropagation through the neighborhood selection decision.

Results

Evaluated on LIAR (12,836 news items, 6 classes) with three label proportion scenarios:

At 10% labeled data: - DHGAT: 0.4503 accuracy, 0.3766 F1-score - Baseline GAT: 0.3904 accuracy, 0.3271 F1-score - Best prior method (GCN): 0.3884 accuracy, 0.3335 F1-score - Improvement: ~4% absolute accuracy over next-best

At 20% labeled data: - DHGAT: 0.4843 accuracy, 0.4213 F1-score - GAT: 0.3962 accuracy, 0.3362 F1-score

At 30% labeled data: - DHGAT: 0.4763 accuracy, 0.3763 F1-score - MGCN: 0.4654 accuracy, 0.4398 F1-score

Per-class analysis shows DHGAT excels at "false," "half-true," and "mostly-true" categories but struggles more with "true" and "pants-fire" (higher misclassification to semantically-close categories). The semantic distance loss term significantly improves performance compared to cross-entropy alone.

Ablation studies on graph structure show: - Speaker + context edges yield highest accuracy. - State and party-affiliation features contribute minimally. - Combining heterogeneous edges outperforms text-only baselines (TextCNN, TextGCN) by 10+ percentage points.

Connections

Notes

Strengths: - Novel framing: treating neighborhood type selection as a learned decision problem is intuitive and well-motivated. - Multi-class setting is more realistic than binary classification and provides finer-grained analysis. - The semantic distance loss that accounts for label orderings is a practical contribution for ordinal fake-news labels. - Solid empirical gains in label-scarce regimes (10–20%), directly relevant to real-world deployment constraints. - Thorough ablation studies on edge-type combinations.

Limitations: - Still a preprint (January 2025); peer-review status unknown. - Computational complexity of running decision network at every node in every layer not quantified. Likely overhead compared to fixed-neighborhood methods. - Graph construction relies on exact speaker profile matching; no handling of typos, aliases, or profile updates. - Evaluation limited to LIAR English dataset. Generalization to other languages, domains, or multimodal content unknown. - Baseline models (TextCNN, TextGCN) use only text; heterogeneous approach has information advantage, making comparison less clean. - Confusion matrix shows systematic error patterns (e.g., "pants-fire" frequently mislabeled as "false") suggesting label imbalance or ambiguity not addressed in the method.