Skip to content
Detecting Bots on Russian Political Twitter

Detecting Bots on Russian Political Twitter

Authors: Denis Stukal, Sergey Sanovich, Richard Bonneau, Joshua A. Tucker

Venue: Big Data, Volume 5, Number 4, 2017 — DOI

TL;DR

Automated Twitter accounts (bots) represent >50% of activity in Russian political discourse. Using supervised ensemble classifiers on 3.8M accounts and 18M tweets from 2014–2015, the authors achieve 95% precision in detecting bots while maintaining >75% recall. Software platform used for tweeting is the strongest bot predictor; bots spike around major political events (Crimean annexation, killing of Boris Nemtsov), suggesting coordinated use for propaganda and news amplification.

Contributions

  • Novel ensemble voting method for bot detection using supervised learning; achieves near-perfect precision while maintaining high recall despite class imbalance.
  • Comprehensive bot behavior analysis over two critical periods in Russian politics (Feb–Oct 2014, Jan–Dec 2015), revealing temporal dynamics and event-driven spikes.
  • Large-scale quantification: 1,068 hand-coded accounts + 230K accounts with ≥10 tweets; demonstrates that ≥67% of politically-active Russian Twitter accounts are bots.
  • Identification of bot activity patterns: characteristic features (platform choice, posting patterns, retweet behavior, URL usage) that distinguish bots from humans.
  • Evidence of bots' role in propaganda: content analysis showing bots amplify headlines and disseminate information about opposition figures and political events.

Method

The authors collected tweets containing 86 politically-relevant keywords/hashtags from Russian-language Twitter accounts during two periods: Feb 6–Oct 1, 2014 and Jan 30–Dec 31, 2015. Dataset: ~3.8M accounts, ~18M tweets total; stratified by keyword threshold.

Labeling: A sample of ~1,000 accounts was hand-coded by undergraduate coders (50 per week, 15 weeks) into 13 account subtypes (e.g., news bots, hashtag gamers, news feeds). These were collapsed into two broad categories: all types of bots vs. humans. High inter-coder reliability (≥75%) ensured quality.

Feature extraction: 42 features capturing account metadata and tweeting activity: - Profile features: default profile image, background image, follower-to-friends ratio, description length, location specification. - Tweeting patterns: hashtag usage, retweet behavior, URL inclination, software platform (Web, Twitter Button, Mobile, Twitterfeed, ifttt.com, etc.), geolocation, time intervals between tweets. - Text-independent (metadata-only); no natural language processing applied to ensure applicability across languages and domains.

Classification: Four classifiers trained separately with 10-fold cross-validation on a balanced training set: 1. Logistic regression with ridge regularization 2. Support Vector Machine (RBF kernel) 3. XGBoost (gradient boosted trees) 4. SAMME (Stagewise Additive Modeling using Multiclass Exponential loss)

Ensemble aggregation: Two voting rules tested: - Majority rule: Account is a bot if ≥6/10 ensemble classifiers predict bot. - Unanimous rule (conservative): Account is a bot only if all classifiers agree.

The majority rule achieved >95% precision with 87% recall on test sets; the unanimous rule maintained 93% precision and 79% recall, favoring precision over recall to minimize false positives.

Results

Prevalence: Across both periods, bots constitute 50–80% of politically-active Russian Twitter accounts tweeting ≥10 times during the study period. Proportion grew from ~40% in early 2014 to >80% by late 2014, then stabilized around 60–70% in 2015.

Temporal dynamics: Sharp increase in bot proportion coinciding with the Crimean status referendum (March 16, 2014) and sustained elevation through the rest of 2014. In 2015, bot activity remained high (~60–70%) with notable spikes around Boris Nemtsov's killing (Feb 28, 2015) and the G20 summit (Sep 5, 2015).

Tweets from bots: ~60–70% of all tweets about Russian politics in the collection originated from bots in both periods. While ratio of tweets-to-accounts is typically larger for bots (1.07–5.20) than humans (1–7.85), bots do not tweet dramatically more frequently than humans on a per-account basis.

Feature importance: Top predictors of bot activity (ridge logistic regression coefficients): 1. Platform: Twitter for Websites (+) 2. Platform: Tweet Button (+) 3. Platform: Twitter for Phone (–) 4. Platform: dwr.it (–) 5. Platform: Twitter for Android Tablets (+) 6. % of tweets with hashtags (–) 7. Platform: Twitter for iPad (–) 8. Platform: Mobile Web (M2) (–) 9. Geo-enabled tweets (binary) (–) 10. Platform: ifttt.com (–)

Software platform dominates: The choice of platform (Web client vs. automated tools like Twitterfeed or ifttt.com) is the single strongest predictor. Bots predominantly post via web interfaces or automated tools; humans favor mobile clients.

Account type switching: A small fraction (75% of ~93K accounts active in both periods; 67% were consistently labeled as bots across classifiers trained on each period independently, suggesting stable bot control rather than frequent hand-offs.

Robustness check: Classifier trained on period 1 predicts period 2 with >80% accuracy, indicating bot detection characteristics are stable over time despite the potential for bot operators to adapt.

Connections

Notes

Strengths: The ensemble voting approach is novel and well-motivated for bot detection: combining multiple classifiers with unanimous or majority voting rules reduces both type I (false positive) and type II (false negative) errors relative to any single classifier. Hand-labeling by multiple coders ensures high-quality ground truth. Focus on metadata-only features (no text analysis) makes the method linguistically and culturally portable. Large-scale application to 230K+ accounts enables population-level estimates of bot prevalence in Russian politics.

Limitations: Bot detection based on observed activity at the time of data collection; the method cannot retroactively classify deleted or suspended accounts, nor can it account for accounts that changed hands (e.g., from bot operator to human). The training set imbalance (even after adaptive sampling) suggests the authors adopted a conservative stance, accepting some false negatives to ensure high precision. Generalizability beyond Russian Twitter unverified; features (platform choice, time zones, account language) are Russia-specific and may require retraining for other contexts. No examination of bot creators' identities or organizational structure.

Significance: This paper provides the first large-scale quantification of bots in Russian political Twitter and establishes that automated accounts are not a marginal phenomenon but comprise >50% of active accounts in political discourse. The temporal correlation between bot activity surges and major political events (Crimea, Nemtsov) strongly suggests bots are used for coordinated propaganda, even if the paper stops short of attributing specific campaigns to state actors (attribution is addressed in Linvill & Warren's later troll factory analysis). The finding that software platform is the strongest bot predictor offers a practical signal for detection in real-time monitoring systems.