Skip to content
Assessing the Russian Internet Research Agency's impact on the political attitudes and behaviors of American Twitter users in late 2017

Assessing the Russian Internet Research Agency's impact on the political attitudes and behaviors of American Twitter users in late 2017

Authors: Christopher A. Bail, Brian Guay, Emily Maloney, Aidan Combs, D. Sunshine Hillygus, Friedolin Merhout, Deen Freelon, Alexander Volfovsky

Affiliation: Duke University (Polarization Lab)

Venue: Proceedings of the National Academy of Sciences, Vol. 117, No. 1, pp. 243–250 (2020) — DOI

TL;DR

Combining longitudinal survey data from 1,239 partisan Twitter users with nonpublic IRA account data, Bail et al. find no evidence that interaction with Russian Internet Research Agency accounts substantially impacted six measures of political attitudes (affective/ideological polarization) or behaviors (political engagement, network composition) over a one-month period in late 2017. Importantly, respondents most likely to interact with IRA trolls were those already highly polarized (strong ideological homophily, high political interest, frequent Twitter use), suggesting the IRA may have failed to polarize because their messages reached primarily entrenched partisans unlikely to change.

Contributions

  • First causal assessment of IRA's political impact: prior work documented IRA content and reach; this is the first study with pre- and post-interaction attitude measurement to assess actual effect on individuals.
  • Longitudinal design with heterogeneous treatment effects: Bayesian causal forest models allow estimation of average treatment effects on the treated (ATT) and heterogeneous effects by individual characteristics; accounts for confounding via propensity weighting.
  • Multiple outcome measures: tests effects across six distinct outcomes (feeling thermometer, social distance toward opposing party, self-rated ideology, liberalism-conservatism scale, number of political accounts followed, ideological composition of Twitter network).
  • Dosage and temporal sensitivity analysis: extends analysis to 16+ months of data and models cumulative exposure (1+, 2+, 3+ interactions) to test whether null effects persist at higher doses or extended time windows.
  • Individual-level characterization: identifies profile of users most likely to interact with IRA—strong echo chambers, high political interest, frequent use—suggesting a mismatch between target characteristics and persuadability.

Method

Sample design: October 2017 survey of 1,239 Republicans and Democrats (recruited via YouGov panel) who use Twitter at least 3 times per week, willing to share Twitter handles, accounts not protected. Stratified to include ~equal numbers of "strong" and "weak" partisans. Followed up in November 2017 with same survey battery.

Key variable: Binary treatment = interaction with IRA accounts between survey waves (either direct engagement: liking, retweeting, mentioning a troll tweet; or indirect: following a troll or being exposed via a friend's mention). Nonpublic data from Twitter's Elections Integrity Hub (4,256 unhashed IRA account identifiers) matched against respondents' public tweet histories.

Outcomes (6 measures): 1. Affective polarization (feeling thermometer): 0–100 rating of opposing party (negative = increased dislike) 2. Affective polarization (social distance): willingness to interact (socialize, work) with opposing-party members 3. Ideological polarization (7-point self-placement): liberal ↔ conservative 4. Ideological polarization (10-item scale): agreement with liberal/conservative policy statements 5. Political engagement (behavior): number of political accounts followed (change between surveys) 6. Network homophily (behavior): % of respondent's political follows that share their party (network drift toward co-partisans)

Statistical model: Bayesian Causal Forest (BCF) — nonparametric Bayesian regression tree model that incorporates propensity weighting to estimate average treatment effects on the treated (ATT) and heterogeneous effects by covariates. Standardized outcomes to mean 0, SD 1 for interpretation.

Control variables: Republican vs. Democrat, Twitter frequency, overall news interest, age, family income, demographics (male, college degree, white, region), treatment assignment in prior field experiment.

Results

Who interacts with IRA?

Binomial regression predicting pre-October 2017 IRA interaction (76 of 1,239 respondents): - Strongest predictor: Ideological homophily (% of political follows that are co-partisans) — users in high-echo-chamber networks ~2 standard deviations more likely to interact - 2nd strongest: Overall political interest (~1.5 SD effect) - 3rd: Twitter frequency (~1.3 SD effect) - Party (Republican slightly higher) not statistically significant

→ Interpretation: Users most vulnerable to troll interaction are precisely those least susceptible to persuasion (already entrenched, highly engaged).

Primary analysis (Oct–Nov 2017, 44 treated, 1,106 control):

Fig. 2 summarizes results: - All 6 outcomes: no significant effect of IRA interaction (95% credible intervals include zero) - Standardized ATT ranges from -0.2 to +0.15, all overlapping zero - No heterogeneous effects by news interest, Twitter frequency, or party affiliation

Dosage and extended timeframe analysis (Feb 2016 – Apr 2018):

Extended analysis using additional YouGov profile surveys (~2 years of follow-up ideology measurements): - Treatment defined as IRA interaction between earliest and latest ideology measurement for each respondent - Tested multiple dose levels: 1+ interaction (213 treated), 2+ (110 treated), 3+ (67 treated) - Both direct-only and direct+indirect engagement operationalizations - Result: No significant effects even with larger treatment group and extended window; if anything, ATTs slightly negative (increasing exposure slightly decreased measured ideology shift), but not statistically significant

→ Suggests null effects are not artifacts of short follow-up or binary treatment operationalization.

Connections

Notes

Strengths: - Rare causal identification: combines pre/post-interaction measurements with nonpublic IRA account data from Twitter, enabling quasi-causal inference (regression discontinuity / propensity-weighted comparison). - Multiple outcomes: tests not just opinion change but also network behavior and political engagement, covering affective and ideological polarization. - Heterogeneous effects analysis: BCF models allow examination of whether effects vary by individual characteristics; transparency about null heterogeneous effects. - Robustness across operationalizations: null findings replicated under less conservative treatment definitions (included pre-treated respondents), extended timeframes, and dosage variations. - Sophisticated propensity modeling: acknowledges that IRA interaction is not random; uses outcome modeling to regularize heterogeneous treatment effects.

Limitations & caveats: - Sample not nationally representative: restricted to active partisan Twitter users (65+ users 3x weekly); excludes independents. May not generalize to less politically engaged populations or other platforms. - Time window: primarily October–November 2017. IRA was more active in 2016 and 2015; unable to assess impact on 2016 election itself (the period of greatest political salience). - Narrow set of outcomes: does not measure influence on news consumption, candidate preference/voting, salience of political issues, media trust, or other domains where influence might operate. Ideology and affect are one piece. - Limited mechanism understanding: study identifies that IRA interaction does not correlate with attitude change but does not explain why — could be lack of causal effect, or could be that effect only emerges at different timescales, through different mechanisms (media coverage of IRA, not direct troll interaction), or for subpopulations not captured. - Measurement limitations: IRA interaction measure includes both direct engagement (likes, retweets) and indirect (following, exposure via friend mention) but cannot verify viewing or viewing duration. Retweets of IRA content by third parties are not captured (Twitter deleted originals). Sensitivity analysis suggests these omissions unlikely to materially change results. - Observational design: even with propensity weighting, cannot rule out unmeasured confounding (e.g., some unmeasured variable correlated with both seeking out IRA content and resistance to persuasion).

Significance & interpretation: This paper is important precisely for its null finding. It contributes to a growing literature suggesting that large-scale information campaigns may have weaker effects on attitudes than anticipated, particularly when directed at already-polarized audiences. The finding aligns with historical political communication research on "minimal effects" (Berelson, Zaller) but tests it in the novel context of algorithmic social media and state-sponsored propaganda.

The paper suggests two non-mutually-exclusive interpretations: 1. IRA failed strategically: Russians targeted entrenched partisans (via algorithmic recommendation or deliberate strategy to amplify existing divisions) rather than persuadable swing voters. 2. IRA effects operate through different mechanisms: propagandists' success may not be measured in direct attitude polarization but in media agenda-setting, offline behavior mobilization, or erosion of institutional trust—none of which this study captures.

The one-paragraph conclusion captures the tension well: "Though we find no evidence that Russian trolls polarized the political attitudes and behaviors of partisan Twitter users in late 2017, these null effects should not diminish concern about foreign influence campaigns."