Study of how language models and other generative AI systems can be used maliciously to create false content at scale. Includes analysis of capabilities for generating fake news, fraudulent emails, impersonation, and coordinated disinformation campaigns.
Overview
As language models improve in fluency and reasoning, concerns grow about their misuse. This topic encompasses research on what tasks language models can accomplish, documentation of actual misuse cases, threat modeling of potential bad actors, and development of detection and mitigation strategies.
Key papers
- 2021-weidinger-ethical-social-risks-language-models — comprehensive taxonomy of 21 risks from language models including malicious uses (disinformation campaigns, fraud, cyberattacks), misinformation harms, and discrimination; foundational reference for LM misuse research
- 2019-solaiman-gpt2-release-strategies — early analysis of GPT-2 misuse potential with threat modeling and partnerships for harm research