Detecting Rumors from Microblogs with Recurrent Neural Networks

Jing Ma; Wei Gao; Prasenjit Mitra; Sejeong Kwon; Bernard J. Jansen; Kam-Fai Wong; Meeyoung Cha

In brief

This paper presents the first deep learning approach for detecting rumors on microblogs, using recurrent neural networks (RNNs) to model the temporal evolution of social media posts. The method learns continuous representations of microblog events without hand-crafted features, outperforming prior work and enabling early rumor detection within 12 hours. The authors release two labeled datasets comprising 5,000+ claims and five million posts from Twitter and Sina Weibo.

Contributions

First application of RNNs to rumor detection on microblogs; demonstrates that RNN-based learning outperforms hand-crafted feature approaches used in prior work.
Explores three RNN architectures: basic tanh-RNN, single-layer LSTM/GRU, and multi-layer GRU; shows that gated units (LSTM, GRU) capture long-distance temporal dependencies better than vanilla RNNs.
Demonstrates efficacy for early detection: achieves 83.9% accuracy on Twitter and 89.0% on Weibo within 12 hours of rumor initiation, substantially earlier than existing methods.
Public release of two benchmark datasets: 992 events (498 rumors) from Twitter, and 4,664 events (2,313 rumors) from Sina Weibo, with full post histories.

Method

The core insight is that rumors generate temporal patterns in posting behavior—questioning, denial, and suspicion cues—that can be learned automatically. Rather than engineering hand-crafted features from user/content/propagation signals, the paper models microblog events as variable-length time series of posts and applies RNNs.

Time series construction: Posts for each event are batched into N equal time intervals (N=50 in experiments). Empty intervals are removed and the timeline is re-partitioned adaptively to capture dense post bursts without excessive sparsity. This yields a fixed-length sequence of variable-duration intervals.

Input representation: Within each interval, the system computes tf–idf scores for the top K vocabulary terms (K=5,000). These scores form the input vector to the RNN.

Model architectures:

tanh-RNN: Basic recurrent unit with tanh nonlinearity.
LSTM/GRU: Gated units with memory cells to capture long-distance dependencies; a 100-dim embedding layer reduces parameter count.
2-layer GRU: Stacked GRU layers for learning higher-level feature interactions.

The output unit applies softmax over two classes (rumor vs. non-rumor); training minimizes squared error via backpropagation with AdaGrad.

Results

On Twitter (992 events, 64% rumors), GRU-2 achieves 88.1% accuracy overall (89.8% F1 for rumors). On Weibo (4,664 events, balanced), GRU-2 achieves 91.0% accuracy (91.4% F1). All RNN variants substantially outperform prior baselines:

SVM-TS (hand-crafted temporal features): 80.8% (Twitter), 85.7% (Weibo)
DT-Rank (enquiry-phrase detection): 64.4% (Twitter), 73.2% (Weibo)
RFC, DTC (hand-crafted user/content/propagation features): 77.2–83.1%

Gated units (LSTM/GRU) outperform basic tanh-RNN (difference of ~4–8%), and 2-layer GRU improves over 1-layer (~2–3%), particularly on noisy Twitter data.

Early detection: Figure 4 shows that RNN-based methods achieve rapid accuracy improvement in the first few hours, stabilizing well before official debunking times. GRU-2 reaches 83.9% (Twitter) and 89.0% (Weibo) accuracy within 12 hours.

Notes

This is a foundational paper in applying deep learning to misinformation detection. The key contribution—learning temporal patterns automatically rather than hand-crafting them—has been influential in later work on rumor and fake-news detection. The public datasets, though modest in size, were among the first benchmark collections for microblog rumor detection and remain useful for comparison.

Strengths: First systematic application of RNNs to this problem; clear ablation studies (tanh vs. LSTM vs. GRU vs. multi-layer); real datasets from two platforms (Twitter and Weibo); empirically strong results including early detection.

Limitations: Feature engineering is simply moved into the embedding and time-series construction—the paper doesn't fully sidestep manual design, only moves it upstream. The time-interval aggregation (Algorithm 1) introduces a hyperparameter (N=50) that may require tuning per platform. Datasets are relatively small by modern standards. No analysis of which temporal signals the RNN learns or why gating helps (interpretability is limited).

Follow-up work has extended this approach with attention mechanisms, graph-based models, and multi-modal data; see Ruchansky et al. 2017 for a more sophisticated graph+RNN hybrid.