Multimodal Misinformation

Multimodal misinformation combines textual and visual content to deceive. It spans two broad categories:

1. Synthetic media: Manipulated images (spliced, inpainted), deepfakes, and generated video—where the visual content is fabricated.
2. Out-of-context: Authentic images paired with false narratives—where the visual content is genuine but the textual framing is deceptive.

The combination of modalities increases persuasiveness and virality: multimodal false claims spread faster and are perceived as more credible than text-alone misinformation (Hameleers et al. 2020). Yet traditional fact-checking and forensic methods often operate on single modalities, missing cross-modal inconsistencies.

Key papers

• Phan et al. (2025) — E-FreeM2: Efficient Training-Free Multi-Scale and Cross-Modal News Verification via MLLMs — Training-free multimodal approach for OOC detection using vision-language models with two-stage chain-of-thought reasoning; achieves 90.0% accuracy on NewsCLIPpings with zero trainable parameters.

• Ai et al. (2026) — The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection — Comprehensive survey of large vision-language models (LVLMs) for multimodal fake news detection; proposes taxonomy of three reasoning paradigms (parameter-frozen, parameter-tuning, reasoning-based); evaluates 10 representative models across 8 datasets.

• Yi et al. (2025) — Challenges and Innovations in LLM-Powered Fake News Detection — Survey of LLM-based multimodal approaches; details text-image-video fusion strategies and identifies multimodal integration as key challenge.

• Raza et al. (2025) — VLDBench: Evaluating Multimodal Disinformation with Regulatory Alignment — Large-scale benchmark (31,339 news articles with images, 500+ hours expert annotation) for multimodal disinformation; shows VLMs outperform text-only baselines by 5–30 percentage points but are vulnerable to cross-modal attacks.

• Qi et al. (2024) — SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection — Vision-language model approach with instruction tuning for OOC detection; generates natural language explanations alongside verdicts.

• Abdelnabi et al. (2022) — Open-Domain, Content-Based, Multi-Modal Fact-Checking of Out-of-Context Images via Online Resources — DT-Transformer combining image-caption encoding with retrieved evidence; demonstrates external evidence retrieval significantly improves multimodal detection.

• Luo et al. (2021) — NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media — Largest OOC detection benchmark (55K samples); establishes dominant evaluation dataset for multimodal misinformation research.

Connections

• Out-of-context detection — specific subproblem of detecting misrepresented authentic images
• Deepfake detection — addresses synthetic media (complementary problem)
• Multimodal fake news detection — broader category including all multimodal approaches
• Vision-language models — primary tools for current multimodal misinformation detection
• Fact-checking — evidence-based claim verification strategies