Skip to content

Exposure to untrustworthy websites in the 2016 US election

Authors: Andrew M. Guess, Brendan Nyhan, Jason Reifler Venue: Nature Human Behaviour, Vol. 4, May 2020, pp. 472–480 — DOI

TL;DR

Using combined survey and web-traffic data from the 2016 US presidential election, Guess et al. found that while 44.3% of Americans aged 18+ visited an untrustworthy website during the campaign, actual exposure was highly concentrated among Trump supporters (56.7% of Trump supporters vs. 27.7% of Clinton supporters). Facebook played a key role in directing people to these sites; however, there was no strong evidence that exposure to factually dubious content affected vote choice or turnout.

Contributions

  • Individual-level exposure measurement — combined representative survey (YouGov Pulse panel) with web-traffic data to measure actual browsing to untrustworthy websites (not just beliefs or social-media sharing)
  • Partisan asymmetry in exposure — documented strong selective exposure: Trump supporters were disproportionately likely to visit untrustworthy websites, reflecting both preference for congenial partisan content and differential exposure via information diet
  • Facebook's amplification role — showed that Facebook was a primary referral source for untrustworthy websites, with heavy Facebook users significantly more likely to consume information from these sites
  • Null finding on electoral effects — despite widespread speculation, found no strong evidence that exposure to untrustworthy websites affected vote choice or voter turnout, suggesting that consumption of false information did not necessarily translate into behavioral changes

Method

The study combined three data sources:

Survey component: YouGov Pulse panel survey administered October–November 2016 to 2,525 respondents who provided informed consent to share web-traffic data. Measured self-reported exposure to untrustworthy websites and demographics (age, gender, race, education, political knowledge, political interest, ideology, candidate preference).

Web-traffic data: Respondents who consented (N=2,170) shared granular records of websites visited via a browser extension or installed software ("digital traces minus passwords"). Data covered 7 October–14 November 2016 during the final weeks of the election campaign.

Untrustworthy website list: Curated list of 108 websites that "publish factually dubious content." Sites were classified by domain-level editorial practices (using Snopes and FactCheck.org databases) and supplemented with analysis of Twitter and social-media traffic patterns to identify hard-news topics.

Outcome measures: 1. Binary exposure — whether respondent visited at least one untrustworthy website 2. Intensity — mean share of overall information diet devoted to untrustworthy websites 3. Electoral outcomes — pre- and post-election survey measures of vote choice intention and self-reported turnout via YouGov-validated voter file linkage

Statistical modeling: OLS regression with survey weights to assess correlates of untrustworthy website exposure. Separate models for binary exposure, information-diet share, and electoral outcomes (vote choice and turnout).

Results

Total consumption: - 44.3% of respondents (N=2,170 with web-traffic data) visited at least one untrustworthy website during the study period - Among those exposed, mean share of information diet from untrustworthy websites: 2.6% overall (2.6% for Clinton supporters, 3.1% for Trump supporters) - Full-sample average across all web pages visited: 0.4% of pages came from untrustworthy websites

Partisan asymmetry (strongest finding): - Trump supporters: 56.7% visited ≥1 untrustworthy website (95% CI = 51.8–61.8%) - Clinton supporters: 27.7% visited ≥1 untrustworthy website (95% CI = 23.1–32.4%) - Adjusted for political knowledge, interest, and education, Trump support remained a strong predictor (β = 0.05, p < 0.05)

Media diet composition: - Untrustworthy conservative websites dominated among Trump supporters (95% CI = 0.7–1.5% of pages) - Untrustworthy liberal websites were negligible for Clinton supporters (−1.1% of pages on hard-news topics) - Pattern suggests asymmetric availability of untrustworthy partisan content favoring pro-Trump disinformation

Facebook's role: - Facebook was a primary referral source: among users who visited at least one untrustworthy conservative website, 59% of visits came via a referring domain (95% CI = 51.8–66.7%); compared with 27.7% for Clinton supporters - Heavy Facebook users were significantly more likely to encounter untrustworthy websites via the platform

Electoral effects: - No significant relationship between exposure to untrustworthy conservative websites and vote choice after controlling for pre-election candidate preference (p ≈ 0.14 for Trump support; p ≈ 0.16 for Clinton support) - No significant relationship with self-reported turnout (p > 0.05 for both groups) - Authors note that post-election survey self-reports of turnout are subject to recall error and social-desirability bias; results suggest either that exposure did not affect the decision or that other factors dominated voting behavior

Connections

  • Related to Prevalence and predictors of fake news dissemination on Facebook — same research team's earlier work on Facebook sharing behavior; this paper extends to exposure via browsing (not just active sharing)
  • Related to [[2020-grinberg-twitter-polarization-election|Polarization and the Twitter Information Ecosystem]] which documents similar asymmetries in partisan content exposure on Twitter
  • Relevant to selective exposure literature — demonstrates empirically that partisan selective exposure operates for untrustworthy websites during an election
  • Extends Social Media and Fake News in the 2016 Election which provided aggregate traffic estimates; this paper adds representative individual-level survey data linked to verified behavioral measures
  • Related to fact-checking and content moderation literatures — discusses limitations of fact-checking websites in reaching those most exposed to false information

Notes

Strengths: - Novel linkage of representative survey to actual web-traffic data overcomes major self-report biases plaguing earlier misinformation research; granular behavioral measure (actual page visits) rather than recalled exposure - Partisan asymmetry finding is striking and well-documented; controlled for political knowledge, interest, education, and ideology - High methodological transparency: detailed lists of untrustworthy websites and statistical specifications in supplementary materials - Null finding on electoral effects is important for tempering overheated claims about fake news determining elections

Weaknesses & limitations: - Exposure ≠ effects; the paper measures consumption, not belief change or persuasion. Some readers who consumed untrustworthy content may have dismissed it; others may have been exposed to counter-messaging elsewhere - Sample composition: respondents who opt into sharing web-traffic data may differ from the general population (higher digital literacy, more tech-savvy, or more ideologically committed on both sides). Sample tilt toward Democrats (42% Clinton, 35% Trump) and higher education - Directionality unclear: Trump supporters preferentially visit untrustworthy websites, but causal mechanism unmeasured — do partisan preferences direct browsing, or does exposure shift preferences? - Cross-sectional design: cannot infer causality from observed correlations or temporal ordering - Electoral outcome measures rely on self-report (turnout) or YouGov's linking to validated voter file (vote choice); validated measure is strong but self-report is vulnerable to post-hoc rationalization

Follow-up questions: - Is the partisan asymmetry in untrustworthy-website consumption driven by differential exposure (algorithmic curation, peer networks) or susceptibility (willingness to visit once exposed to a link)? - Does fact-checking on these untrustworthy websites change minds among visitors, or do visitors systematically dismiss corrections? - How much of the null electoral effect is due to small sample size, publication bias favoring null results, or genuine ineffectiveness of exposure on behavior change? - Do intervention studies (randomized exposure to corrections, media literacy training) show larger effects than observational data?

Significance: This paper provides systematic empirical evidence that consumption of untrustworthy websites during the 2016 election was real but selective, concentrated among partisan Trump supporters, and often channeled via Facebook. The null finding on electoral effects is important: it complicates the "fake news won the election" narrative while leaving open the question of whether false information affected specific voters or swing constituencies. The work is foundational for understanding the relationship between exposure to low-credibility information and actual behavior change in electoral contexts.