Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models

Authors: Jiaying Wu, Fanxiao Li, Zihang Fu, Min-Yen Kan, Bryan Hooi

Venue: ICLR 2026 — arXiv:2505.15489

TL;DR

This paper introduces DECEPTIONDECODED, a benchmark of 12,000 image-caption pairs for evaluating multimodal misinformation detection with focus on creator intent. The authors find that state-of-the-art vision-language models struggle with intent reasoning and rely on superficial cues, motivating a framework for intent-aware multimodal misinformation detection that systematically synthesizes intent-level reasoning.

Contributions

• DECEPTIONDECODED dataset: A large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, with controlled intent-guided manipulations spanning multiple communication intents and misinformation strategies.
• Intent-aware framework: Proposes a conceptualization of creator intent along two dimensions—desired influence (target societal impact) and execution plan (how to achieve it)—grounded in communication strategy theory.
• Comprehensive benchmark of VLMs: Evaluates 14 state-of-the-art vision-language models on three intent-centric tasks: misleading intent detection, misleading source attribution, and creator desire inference.
• Systematic evaluation: Reveals that current VLMs are fragile at intent reasoning and overly reliant on superficial alignment cues, surface-level polish, and heuristic authenticity signals.

Method

The framework operationalizes creator intent as a two-dimensional construct combining desired influence (e.g., "public health and safety", "political polarization") and execution plan (open-ended description of implementation strategy).

Data generation pipeline: 1. Source trustworthy news articles from VisualNews (Liu et al., 2021) filtered for high-impact journalistic domains (politics, disasters, health). 2. For each article, use GPT-4o to simulate both malicious and trustworthy content creators with specified intent configurations. 3. Generate intent-aligned textual captions and visual modifications (subtle or significant distortions, background edits) using GPT-4o and FLUX.1 image synthesis. 4. Validate data quality through human evaluation of (a) binary misleading labels (99.2% accuracy for text; 89.2% for images), (b) data realism (98%+ for text; 91%+ for images), and (c) faithful intent alignment (92% for text; 86% for images).

VLM evaluation: Benchmark 14 models (GPT-4o, GPT-4o mini, Claude-3.5 Haiku, QwenVL, LLaVA, etc.) on three intent-centric tasks. Analysis reveals models struggle to recognize misleading intent, often relying on shallow consistency checks and heuristic confidence signals rather than deep reasoning about creator communicative objectives.

Results

VLMs show variable performance across intent-detection tasks. GPT-4o achieves 66.0% accuracy on misleading intent detection; most other models perform substantially worse (e.g., QwenVL-7B at 53.5%). Models perform even more poorly on adversarial image-edited versions, with accuracy drops of 10–20 percentage points. Fine-tuning on DECEPTIONDECODED demonstrates transferability to real-world multimodal misinformation detection, with consistent improvements across three downstream MMD benchmarks.

Visual manipulation proves particularly challenging: edited images with Nano Banana Pro result in near-perfect synthesis quality (consistent intent realization across both dimensions), yet current models still misdetect intent in 30–40% of cases.

Connections

• Related to Misinformation and fake news detection and Visual misinformation as core detection challenges.
• Builds on Multimodal Learning frameworks for joint image–text understanding.
• Extends Vision-language models evaluation toward robustness and intent-aware reasoning.
• Complements Fact-checking and corrections and Content moderation by targeting creator communicative objectives rather than just factual accuracy.
• Cites and benchmarks against FakeNewsNet and other multimodal MMD benchmarks.

Notes

Strengths: The focus on creator intent is novel and well-motivated; the intent-guided simulation framework grounded in communication theory provides a principled approach to controlled multimodal misinformation generation. The dataset's scale (12K) and methodological rigor (human evaluation protocols, faithfulness checks) are substantial. Extensive evaluation of 14 VLMs surfaces important fragility patterns.

Weaknesses: The framework, while principled, simplifies real-world creator objectives to two dimensions; nuance in adversarial intent may be lost. The paper raises important questions about VLM robustness but does not yet provide a ready solution—fine-tuning transfers to downstream tasks but margins are modest. The dataset release is restricted to verified researchers under binding usage agreements, limiting broader reproducibility and follow-up work.

Open challenges: Fully open-ended intent articulation and evaluation remains an open problem. The paper hints that adaptive prompting strategies (e.g., LLM-as-a-Judge paradigms) may help; this deserves deeper exploration. How intent detection scales to highly subtle manipulations or novel creator strategies remains unclear.