Fake News Detection on Social Media using Geometric Deep Learning

Authors: Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, Michael M. Bronstein

Venue: arXiv, 2019 — arXiv:1902.06673

TL;DR

A graph neural network approach for fake news detection that exploits social network structure and news propagation patterns rather than content alone. The method integrates user profiles, social connections, and cascade dynamics to achieve 92.7% ROC AUC on Twitter data and can detect fake news within hours of propagation.

Contributions

• Novel application of geometric deep learning (graph convolutional networks) to fake news detection that naturally unifies content, social context, and propagation features
• Graph-structured representation encoding user demographics, activity, network topology, and news spreading patterns
• Demonstration that propagation-based features are language-independent and robust to adversarial attacks compared to content-based methods
• Empirical evidence that fake news exhibits reliably detectable propagation signatures within 1–2 hours of initial spread

Method

The paper proposes a four-layer graph convolutional neural network (GCN) architecture with two convolutional layers followed by mean pooling and fully connected layers for binary classification.

Graph construction: For each news story (URL), the model represents cascades as directed graphs where nodes are tweets and edges encode spreading relationships. Two annotation strategies are used:

1. URL-wise: All tweets spreading a URL form a single graph; the model predicts whether the URL contains fake news
2. Cascade-wise: Each cascade (retweet chain) is a separate graph; the model predicts per-cascade veracity

Feature extraction: Nodes (tweets/users) and edges are characterized by four feature categories:

• User profile: Geolocation, verification status, account age
• User activity: Favorites, retweets, list memberships, account status
• Network and spreading: Social connections (followers/friends), cascade tree structure, retweet timestamps, membership in shared cascades
• Content: GloVe word embeddings, hashtags

Spreading trees: News diffusion paths are estimated by inferring social connections and timestamps. If user \(u_i\) follows user \(u_j\) who posted the source tweet \(t_j\), news spreads from \(t_j\) to \(t_i\) at \(u_i\)'s timestamp. Conservative heuristics handle cases where direct follows are absent.

Training: The model uses hinge loss and is trained on ~1,000 labeled Twitter cascades (2013–2018) verified by professional fact-checking organizations (Snopes, PolitiFact, BuzzFeed).

Results

URL-wise classification (all cascades aggregated): - 92.70 ± 1.80% ROC AUC (5-fold cross-validation) - 83.26% true label fraction, 16.74% false labels

Cascade-wise classification (per-cascade predictions): - 88.30 ± 2.74% ROC AUC - 81.73% true labels, 18.27% false labels (minimum 6 tweets per cascade)

Minimum cascade size: Performance saturates at cascades with ≥6 tweets; using this threshold yields the best balance of coverage and accuracy.

Feature importance (ablation study): - User profile and network/spreading features are most important (~90% ROC AUC when combined) - Content features contribute marginally (4% performance drop when removed) - Language independence of propagation-based features suggests generalization across languages

Temporal dynamics: - Fake news detectable reliably within 1–2 hours of initial spread (reaching >90% ROC AUC in URL-wise setting) - Performance in cascade-wise setting plateaus after 7 hours (~87% AUC) - Average cascade size is 2.79 tweets; cascades follow highly variable power-law distribution

Model aging: When trained on historical data and tested on temporally disjoint data, URL-wise model exhibits slight deterioration after 180 days (~5% AUC drop), while cascade-wise model remains stable (4% drop after 260 days). Authors attribute cascade robustness to simpler topological patterns less affected by evolving user behavior.

Connections

• Related to Shu Hierarchical Propagation and Shu Social Context Trifn in exploiting network structure for detection
• Extends Ruchansky Csi (which uses GCN on rumor cascades) to heterogeneous features combining content and structure
• Complements content-based methods like Wang Eann Multimodal by demonstrating propagation patterns as orthogonal detection signal
• Cited by follow-up work on graph-based detection and adversarial robustness of network features

Notes

Strengths: - Principled integration of multiple feature modalities via graph structure; avoids ad-hoc feature engineering - Propagation-based approach is inherently language-agnostic and potentially robust to linguistic adversarial attacks - Early detection capability (1–2 hours) is practically valuable for intervention - Temporal analysis provides insight into model degradation—useful for practitioners deploying in production - Clear ablation studies quantify the contribution of each feature category

Weaknesses: - Dataset relatively small (~1,000 labeled cascades across 150 users and 1,084 unique URLs) and limited to Twitter; generalization to other platforms unclear - Cascade-wise setting assumption (one cascade per URL) is unrealistic; URLs often spread through independent cascades that may require aggregation heuristics - Social graph inference from follower/followee relationships is conservative and may miss actual diffusion paths, especially for users outside the crawled network - Model aging after 180 days suggests temporal distribution shift; authors acknowledge but don't address root causes (e.g., evolving bot behavior, new disinformation campaigns) - Limited comparison to strong baselines; primarily compared to variants of their own method, not state-of-the-art content-based or other propagation-based approaches

Interesting findings: - The observation that only ~2 hours of propagation suffices for reliable detection suggests fake news may have inherent signature features early, before large cascades form - Polarization analysis (Figure 4) showing credible and unreliable users form distinct network clusters is suggestive of structural homophily in misinformation ecosystems - The resistance of cascade-wise detection to aging (vs. URL-wise) hints that local network patterns are more stable over time than global user behavior trends

Future directions: - Cross-platform validation and transfer learning to platforms with different user behavior (Facebook, Reddit, WhatsApp) - Adversarial robustness: can the model be fooled by synthetic cascades or coordinated inauthentic behavior? - Integration with content features; the paper ignores tweet text except for word embeddings—linguistic analysis may provide complementary signals - Interpretability: which specific network motifs or user features drive classification decisions?