CSI: A Hybrid Deep Model for Fake News Detection

Authors: Natali Ruchansky, Sungyong Seo, Yan Liu Venue: CIKM'17, September 2017 — DOI

TL;DR

CSI is a three-module deep learning model that detects fake news by jointly modeling article text, temporal patterns of user engagement (response), and user behavior patterns (source). The Capture module uses LSTM to learn temporal response signals; the Score module learns user suspiciousness from co-engagement patterns; the Integrate module combines both for classification. On Twitter and Weibo, CSI achieves 89.2% and 95.3% accuracy respectively, outperforming prior work while using fewer parameters.

Contributions

• First model to explicitly capture all three commonly cited fake news signals: text, response (temporal engagement patterns), and source (user behavior).
• Neural network-based approach that avoids hand-crafted features and makes no distributional assumptions about user behavior.
• Produces interpretable outputs: article veracity predictions, user suspiciousness scores, and low-dimensional representations of both users and articles usable for independent analysis.
• Achieves state-of-the-art accuracy on Twitter and Weibo with fewer parameters than competing RNN-based methods (52K vs. 621K for GRU-2).

Method

Three-module architecture:

The Capture module processes temporal engagement sequences for each article. For each time partition, it constructs features combining (1) engagement frequency η and inter-engagement time Δt; (2) user features xu derived from SVD of a user-article co-engagement matrix; (3) doc2vec embeddings xτ of textual content. These are passed through an embedding layer then fed to an LSTM, which outputs a low-dimensional article representation vj capturing temporal and textual response patterns.

The Score module extracts user representations from an implicit user graph where edges reflect co-engagement on articles. User-feature vectors are computed via SVD on the weighted adjacency matrix, then passed through a fully connected layer to produce both a vector representation ỹi and scalar suspiciousness score si for each user. The score measures propensity to participate in group-promoted fake content.

The Integrate module masks the user scores to select only those users who engaged with a given article aj, averaging to produce pj (aggregate user suspiciousness for that article). This is concatenated with vj and passed through a final fully connected layer with sigmoid output to predict article veracity.

Training and features:

Binary cross-entropy loss with L2 regularization on user-layer weights. Random dropout on Wa and Wr to reduce overfitting. For feature extraction: SVD rank 20 (Twitter) or 10 (Weibo) for user features xu; SVD rank 50 for user graph features yi; embedding and hidden dimensions set to 100. Temporal granularity: hourly partitioning of engagement sequences. Text: doc2vec on original language (with Chinese segmentation for Weibo).

Results

Classification accuracy:

CSI achieves 89.2% accuracy (F-score: 0.894) on Twitter and 95.3% accuracy (F-score: 0.954) on Weibo, significantly outperforming five baselines including SVM-TS, DTC, LSTM-1, and GRU-2. Ablation studies show: (1) adding temporal features to text improves accuracy >1%; (2) adding user features provides 4.3% gain over GRU-2. CSI maintains >80% accuracy with only 10% of labeled training data.

User representations:

The Score module's outputs correlate with ground-truth proxy measures. User suspiciousness score si correlates 0.525 (Twitter) and 0.485 (Weibo) with the fraction of fake news a user engages with (ℓi). Users flagged as highly suspicious (high si) concentrate their activity on fake articles; low-suspicion users engage broadly. User representation vectors ỹi exhibit clear separation in 2D projection, with distance between user pairs correlating with differences in fake-news engagement fraction.

Temporal behavior analysis:

Suspicious users are among the first to promote fake articles, with shorter lag times between article publication and engagement. Suspicious users show burst patterns of rapid engagements, a behavior less pronounced for true news and for low-suspicion users.

Article representations:

The Capture module's output vectors vj can be used for unsupervised article clustering, revealing natural separation between different types of fake news (e.g., satire vs. spam) as well as true news.

Connections

• Related to User Propagation Based Detection via temporal engagement analysis of user behavior on articles.
• Extends Rumor Detection Temporal by incorporating multi-modal features (text, temporal response, user behavior) in a unified deep model.
• Contrasts with Text Only Fake News Approaches by showing joint modeling of all three characteristics outperforms single-modality methods.

Notes

Strengths:

The paper makes a clear empirical case that combining text, response, and source signals improves accuracy substantially. The modular design is elegant: each module can be analyzed independently, producing interpretable outputs (user scores, article representations) beyond just classification. The lack of distributional assumptions and domain-specific feature engineering makes the approach general. Parameter efficiency (52K) is notable for a deep model and facilitates training with limited labeled data.

Limitations and open questions:

Ground truth for user labels is unavailable; the proxy measure (fraction of fake articles engaged with) conflates user intent and awareness. The model relies on historical engagement data and social network structure; performance on emerging stories without rich engagement is unclear. No analysis of temporal generalization—do learned patterns transfer across time windows or to new platforms? The interaction between text and temporal/source signals is implicit in the joint training; explicit analysis of signal independence or redundancy would strengthen the work. How does the model perform on coordinated inauthentic behavior (e.g., bot networks with non-obvious engagement patterns)?