Skip to content

From Fake News to #FakeNews: Mining Direct and Indirect Relationships among Hashtags for Fake News Detection

Authors: Xinyi Zhou, Reza Zafarani, Emilio Ferrara Venue: ACM Conference, 2022 — arXiv

TL;DR

Hashtags reveal news credibility signals during propagation: fake news clusters with hashtags used primarily for misinformation, while true news uses hashtags associated with reliable content. The authors propose NewsTag, a language-independent semi-supervised algorithm that mines both direct co-occurrence and indirect relationships among hashtags via an all-relations graph, achieving 73% macro F1 on ReCOVery and 85% on MM-COVID datasets, with strong early-detection performance.

Contributions

  • First work to investigate indirect relationships among hashtags for fake news detection through an iterative graph-based approach that generalizes to any homogeneous network
  • Language-independent semi-supervised algorithm enabling multilingual fake news detection with limited labeled data
  • Empirical validation showing hashtags are a stronger signal than user posts and content features alone, especially at early propagation stages
  • Demonstrates that true news exhibits higher social media popularity than fake news in early hours, supporting graph-based credibility inference

Method

The method constructs an all-relations hashtag graph \(G = (V, E, W_{\text{all}})\) where nodes are hashtags co-occurring in posts about the same news article. Direct relationships capture hashtag co-occurrence weights; indirect relationships are computed iteratively:

\[W_{\text{all}} = \lim_{k \to \infty} (N + N^2 + \cdots + N^k) = N(I - N)^{-1}\]

where \(N = W_{\text{dir}} / \max(\sum_l w_{kl})\) normalizes the direct weight matrix.

For each hashtag \(h_k\), an initial credibility score \(c_k^0\) is computed as the weighted average credibility of news articles containing it:

\[c_k^0 = \frac{\sum_i y_i I(h_k \in H_i)}{\sum_i I(h_k \in H_i)}\]

Scores are optimized via a convex cost function balancing graph-based smoothness with labeled data fidelity, solved iteratively:

\[c(k_2) = \mu X c(k_2-1) + (1-\mu)c^0\]

where \(X = D^{-1/2}W_{\text{all}}D^{-1/2}\) and \(\mu \in (0,1)\) is a regularization parameter. News credibility is inferred by aggregating hashtag credibility scores from its dissemination posts.

Results

General performance (80% train / 20% test): - ReCOVery: 0.73 macro F1, 0.81 micro F1 - MM-COVID: 0.85 macro F1, 0.86 micro F1

NewsTag outperforms content-based baselines (LIWC, RST) by ~20% and propagation baselines using tweet embeddings (T2VC, T2VW) by ~10% on both datasets. Variants without indirect relationships (NewsTag\I) or without edge weights (NewsTag\W) degrade performance by ~3%, validating both components.

Early detection: With only 20% training data, NewsTag achieves 0.66 macro F1 on ReCOVery (vs. 0.50 for LIWC). When limited to 12 hours post-publication, it maintains 0.62 macro F1 (vs. 0.45 for content baselines), demonstrating viability for rapid credibility assessment.

Case study: Visualization of the hashtag graph shows conspiracy-related hashtags (#saynotovaccine, #5gdeath, #deepstate, #plandemic) cluster with extreme credibility scores (≤−0.9), validating the graph's ability to identify coordinated inauthentic behavior.

Connections

Notes

Strengths: - Language independence is significant; method requires only hashtag co-occurrence, not content NLP - Early detection results (12–60 hours) are practically valuable for real-time systems - Graph construction is elegant and generalizable to any node-relationship network - Evaluation on two independent COVID-19 datasets with different linguistic contexts strengthens claims

Weaknesses: - Limited to platforms with rich hashtag adoption (Twitter); applicability to other social media unclear - No analysis of adversarial hashtag manipulation or coordinated account strategies - Comparison to more recent deep learning baselines (e.g., GNNs) is absent; T2VC/T2VW are from 2016 - Assumes hashtag credibility is stable across topics, but highly polarized hashtags may flip credibility depending on context

Questions for follow-up: - How does performance scale to non-COVID topics with less hashtag coherence? - Can indirect relationships capture coordinated manipulation (intentional credibility flipping)? - Does cross-lingual hashtag overlap (same hashtag used in multiple languages) help or hurt multilingual detection?