Assessing the risks of 'infodemics' in response to COVID-19 epidemics

Authors: Riccardo Gallotti, Francesco Valle, Nicola Castaldo, Pierluigi Sacco, Manlio De Domenico

Venue: Nature Human Behaviour, Vol. 4, December 2020, pp. 1285–1293 — DOI

TL;DR

Analysis of 112+ million geolocated Twitter messages about COVID-19 (January–March 2020) across 64 languages and countries. Develops an infodemic risk index (IRI) measuring exposure to unreliable news; demonstrates that waves of unreliable information precede pandemic waves, providing early-warning signals for policy intervention. Reliable information sources become dominant as crises escalate.

Contributions

• Large-scale global Twitter dataset (112M messages) with geographic and temporal granularity across 64 languages and countries during early pandemic
• Novel "infodemic risk index" (IRI) framework quantifying per-capita exposure to unreliable news, accounting for verified/unverified user fractions and heterogeneous information consumption
• Empirical validation that infodemic waves precede confirmed-case waves with 2–5 day lead time, offering early-warning capability for public health intervention
• Comprehensive categorization of news sources into nine reliability tiers (science, mainstream media, satire, clickbait, political, fake-or-hoax, conspiracy, shadow, other), with manual validation of 4,000+ URLs
• Country-level analysis showing high heterogeneity in infodemic risk — some countries (Russia, Germany) with low risk; others (Iran, Italy) with high risk — correlated with socioeconomic factors and political communication patterns

Method

The authors collected 112M Twitter messages between January 22 and March 10, 2020 using keywords (coronavirus, ncov, #Wuhan, covid19, sars-cov-2, covid) across all languages. Each message was geolocated using user self-declared location strings (ArcGIS API), filtered to ~50% accurate at country level. To address Twitter's demographic limitations (younger, educated, over-represented in English), they:

1. Classified websites — Manually checked 4,000+ domains referenced in collected tweets using Media Bias/Fact Check and other sources. Assigned URLs to nine categories: Science (validated via scientific scrutiny), Mainstream Media (professional fact-checking and editorial standards), Satire (explicitly humorous misinformation), Clickbait (attention-grabbing distortion), Political (partisan representation), Fake or Hoax (manipulative fabrication), Conspiracy and Junk Science (implausible conceptualizations with pseudo-scientific methods), Shadow (URL shorteners), Other (unclassifiable).

2. Developed the Infodemic Risk Index (IRI) — For a generic user \(u\) in time window \(\Delta t\), the exposure to unreliable news is: $\(\text{IRI}(t, t + \Delta t) = \sum_i p\text{IRI}_i(t, t + \Delta t)\)$ where \(p\text{IRI}_i\) is the partial infodemic risk for each category of users \(i\) (verified users, unverified users). The formula accounts for the fact that unverified accounts (and bots) reach dramatically fewer users than verified users with large follower counts. The index ranges from 0 (no infodemic risk) to 1 (maximum infodemic risk).

3. Temporal and geographic analysis — Plotted IRI evolution daily for countries; compared with epidemiological case counts from WHO and JHU datasets; tested for temporal precedence using cross-correlation and Granger causality.

4. Filtering and limitations — Excluded messages with ungeocoded locations, non-English URLs, and API-access-limited periods. Acknowledged that Twitter analysis captures only English-language speakers and over-samples wealthy nations; results are not representative of offline communication or non-English-speaking populations.

Results

Infodemic evolution precedes confirmed cases: In Italy, infodemic risk spiked (IRI = 0.9) weeks before case surges. In the United States, IRI remained lower (≈0.4–0.5) but showed similar temporal structure. Across multiple countries, verified influencers with large followings amplified reliable information reliably, while unverified accounts distributed mixed-credibility content, contributing to overall risk at scale.

Geographic heterogeneity: Countries with higher infodemic risk correlated with political polarization, lower socioeconomic development indices (HDI), and historical media trust gaps. Russia and Germany exhibited low infodemic risk despite high case counts; Iran and Italy showed high risk despite moderate caseloads. The relationship between epidemic and infodemic is non-linear.

Reduction in infodemic risk as crisis escalates: As COVID-19 became mainstream and institutional responses solidified (mask mandates, lockdowns, vaccination campaigns), the fraction of reliable-source messages increased. The paper interprets this as evidence that verified influencers (journalists, public health officials, fact-checkers) shifted focus toward credible information sources, and media literacy interventions may have had cumulative effects.

Source reliability distribution: Science and Mainstream Media domains dominated early discussion but accounted for only ~40% of total message volume; Conspiracy and Junk Science, Fake or Hoax, and Political content combined represented ~25% of messages, with highly volatile daily fluctuations.

Connections

• Extends foundational Infodemic framework with quantitative risk metrics and predictive temporal structure
• Complements Cinelli et al. (2020) platform-comparative analysis by focusing on geographic and temporal dynamics within Twitter
• Related to Singh et al. (2020) myth-debunking effort and Memon & Carley (2020) community characterization
• Contributes to COVID-19 Misinformation and Infodemic literature on information dynamics and early-warning signals
• Methodologically informs Social Media Analysis methodologies for information credibility assessment

Notes

Strengths: real-time global dataset, novel IRI metric capturing heterogeneous user influence, strong temporal precedence evidence suggesting policy-actionable lead time for intervention, manual validation of source reliability. Geographic visualization clearly illustrates heterogeneous infodemic manifestation worldwide.

Limitations: Twitter over-samples wealthy English-speaking nations and young users; results are not representative of global communication. Manual URL classification on 4,000+ sites is labor-intensive and may reflect subjective bias in categorization. Geolocation accuracy ~50% at country level introduces measurement error. The study does not directly measure individual susceptibility to misinformation or behavioral outcomes — only exposure via social network topology. Does not separate information quality from individual verification behavior (e.g., fact-checking on external sources). Causal inference of information's role in epidemic spread is suggestive rather than definitive.

Despite limitations, the temporal precedence of infodemic waves over confirmed-case waves is striking and suggests that early monitoring of information credibility could inform public health communication strategies — supporting information campaigns by credible sources, reducing amplification of conspiracy narratives, and targeting media literacy interventions to high-risk groups.