Large language models (LLMs) and other generative AI systems have introduced new capabilities and risks for misinformation, disinformation, and influence operations. These systems can generate persuasive, contextually appropriate false claims at scale, tailor content to specific audiences, evade detection through stylistic heterogeneity, and—when integrated into multi-agent architectures—coordinate autonomously to manipulate discourse.
Key capabilities that elevate risk include:
- Scale and cost: LLMs generate propaganda and disinformation inexpensively and without human bottlenecks
- Personalization: Conditional generation enables tailored messaging to specific audiences, communities, or belief systems
- Coherence: LLMs maintain consistent narratives across multiple accounts and platforms
- Sophistication: Techniques like chain-of-thought prompting refine LLM reasoning and can be repurposed to generate more convincing falsehoods
- Multi-agent coordination: LLMs integrated into autonomous agent systems enable swarm-like behavior with decentralized coordination and adaptive learning
Key papers
- 2026-schroeder-malicious-ai-swarms — comprehensive threat assessment of LLM-powered multi-agent systems deployed as coordinated influence swarms; describes technical capabilities (decentralized coordination, infiltration, human-level mimicry, self-optimization), harm pathways (synthetic consensus, training-data poisoning, harassment), and governance/detection strategies
- 2023-goldstein-language-models-influence — language models as a new vector for automating propaganda and influence campaigns
- 2021-weidinger-ethical-social-risks-language-models — taxonomy of risks from language models including detailed treatment of misinformation harms
Related topics
- influence-operations (primary application)
- deepfakes (multimodal synthesis)
- propaganda (content type)
- misinformation (false claims)
- detection-methods (technical defenses)