Large language models and misinformation

Large language models (LLMs) like GPT-3, GPT-4, and other transformer-based models present both opportunities and challenges for misinformation research. LLMs can be applied to detection, intervention, and attribution, but they also enable sophisticated misinformation generation. This topic covers research on using LLMs for misinformation detection and understanding their limitations and failure modes.

Detection and verification

Recent work uses LLMs for misinformation detection by prompting them to evaluate claim truthfulness or using them as evidence retrievers in fact-checking pipelines. 2023-pelrine-reliable-misinformation-mitigation demonstrates that GPT-4 achieves better performance than RoBERTa on binary and soft classification tasks across multiple datasets and languages, with uncertainty quantification enabling systems to gracefully decline classification when context is insufficient.

Challenges and limitations

LLMs present challenges:

• Knowledge cutoff: Performance degrades on claims beyond the training cutoff
• Hallucination: LLMs can generate plausible but false evidence
• Prompt sensitivity: Classification outcomes vary significantly across prompting strategies
• Multilingual degradation: Performance often drops sharply in languages other than English
• Calibration: Raw probabilities from LLM outputs may not reflect actual confidence

LLM-generated misinformation

LLMs also enable generation of sophisticated misinformation at scale. Research shows that LLM-generated misinformation is harder for both humans and detectors to identify than manually-written content with equivalent semantics. 2024-chen-llm-generated-misinformation-detection documents this challenge.

Key papers in this wiki

• 2023-pelrine-reliable-misinformation-mitigation — Evaluates GPT-4 for classification on LIAR and new LIAR-New dataset; shows uncertainty quantification improves performance by enabling systems to refuse impossible examples
• Quelle et al. (2023) — Surveys LLM applications to fact-checking and identifies research gaps
• Chen et al. (2024) — Shows LLM-generated misinformation is harder to identify than human-written equivalents
• Liu et al. (2023) — Comprehensive survey of trustworthiness in LLMs including robustness to adversarial inputs

Related topics

• misinformation-detection — broader framework for detection methods
• fact-checking — fact verification workflows
• generalization — model generalization across datasets and domains
• uncertainty — uncertainty quantification in model predictions