Disinformation 2.0 in the Age of AI: A Cybersecurity Perspective

Authors: Wojciech Mazurczyk, Dongwon Lee, Andreas Vlachos Venue: ACM Publication, August 2023 — arXiv:2306.05569

TL;DR

Disinformation should be treated as a cybersecurity threat, particularly as AI advances enable its creation and dissemination at unprecedented scale. The paper introduces "disinformation 2.0"—where generative AI, perturbation techniques, detection evasion, and optimized spread strategies make false content more persuasive and harder to detect. The authors propose a defense-in-depth approach with four layers: social network, ISP, device, and user.

Contributions

• Establishes the case for treating disinformation as a formal cybersecurity threat
• Defines "disinformation 2.0" and four plausible attack scenarios enabled by AI
• Proposes a layered defense architecture inspired by classical cybersecurity
• Identifies specific countermeasures at each layer and their information-flow interactions

Method

The paper frames disinformation detection and defense through a cybersecurity lens. It argues that just as cybercrimes (scams, theft) migrated to digital platforms, disinformation has evolved similarly—but AI now enables creation and dissemination at scale. The authors then adapt a well-established security principle: defense-in-depth.

Four attack scenarios for disinformation 2.0: 1. Adversaries use generative AI (ChatGPT, DALL-E) to create more convincing false content 2. Adversaries use AI perturbation techniques to subtly modify existing content, evading detection 3. Adversaries attack detection systems themselves, promoting disinformation and demoting genuine news 4. Adversaries use AI to strategize dissemination, maximizing network impact while avoiding bot detectors

Defense layers: - Social Network-level: detection, spread control, source verification, whitelist/blacklist, algorithmic bias mitigation - ISP-level: filtering and blocking verified disinformation domains (like phishing controls) - Device-level: in-browser/app detection of deepfakes and AI-generated text, source verification, alerts on suspicious sharing - User-level: education, prebunking, empowering citizen journalists with technical tools

Results

The paper does not present empirical results but rather a conceptual framework. It emphasizes that no single defense is sufficient; all layers must operate in tandem with diverse detection approaches. The framework highlights information flow between layers—e.g., social network detection can tag content for device-level verification; device-level discoveries can feed back to platforms.

Connections

• Related to Fake content detection via detection evasion and adversarial robustness
• Extends Deepfakes threat model to include strategic dissemination
• Aligns with cybersecurity defense-in-depth principles (defense across multiple network layers)
• Complements Content moderation and social platform governance
• Related to Misinformation interventions via prebunking and user education

Notes

This position paper makes a valuable contribution by framing disinformation as a cybersecurity problem and importing proven defensive principles (layered defense, diverse mechanisms, information sharing). The four attack scenarios are plausible and concrete. However, the paper lacks empirical evaluation—the framework is conceptual. Implementation challenges (privacy, false positives, cross-platform coordination) are acknowledged but not deeply explored. The emphasis on all four layers operating in tandem is realistic but may overestimate practical deployment coordination.