
Contributions
- Comprehensive taxonomy of hallucination in NLG: intrinsic vs. extrinsic hallucinations and source-divergence mechanisms
- Systematic review of hallucination sources: data quality issues, training/inference misalignments, exposure bias, parametric knowledge bias, and inherent divergence in open-domain tasks
- Comparison of four metric classes: statistical (lexical overlap), model-based (information extraction, QA, NLI-based, language model entailment), human evaluation, and faithfulness classification approaches
- Task-specific analysis of hallucination in eight NLG applications: abstractive summarization, dialogue generation, generative QA, data-to-text, machine translation, image captioning, and LLMs
- Review of mitigation strategies organized by two categories: data-related methods (building faithful datasets, cleaning, augmentation) and modeling/inference methods (architecture choices, reinforcement learning, controllable generation, post-processing)
- Research challenges and future directions: fine-grained hallucination detection, automatic fact-checking for extrinsic hallucinations, generalization across tasks and languages, and human-model evaluation alignment
Method
The survey is organized in three parts: (1) a general overview of hallucination metrics, mitigation methods, and contributors to hallucination; (2) task-specific analysis of hallucination in eight NLG applications; and (3) hallucination challenges in large language models.
Definitions: The paper distinguishes hallucination as the psychological term (perceiving without stimulus) adapted to NLG as models generating unfaithful or nonsensical text relative to source content. Two types emerge:
- Intrinsic hallucinations: Generated text contradicts or misuses information from the source.
- Extrinsic hallucinations: Generated text contains information not mentioned in and not inferable from the source—essentially fabricated facts.
Contributors to hallucination:
- Data — source-reference divergence (misaligned training pairs, heuristic collection); heuristic dataset construction; insufficient source coverage
- Training and inference — defective encoder representation; erroneous decoder generation through wrong entity associations, randomness from sampling strategies, exposure bias between training and inference; exposure bias (teacher forcing vs. model-generated history at inference); parametric knowledge bias (models relying on pre-training knowledge over input); inherent divergence in tasks with subjective, multi-faceted outputs (open-domain dialogue, abstractive summarization)
Metrics: Four classes evaluated:
- Statistical metrics (lexical n-gram matching ROUGE, BLEU, METEOR) — simple but insensitive to semantic variations
- Model-based metrics (Information Extraction–based, QA-based, NLI-based, Language Model–based entailment) — more robust to paraphrasing but computationally expensive and prone to errors from subcomponents
- Human evaluation — gold standard but labor-intensive; challenges include scorer disagreement, cost, and coverage limitations
- Faithfulness classification metrics — emerging approach using datasets annotated for hallucination presence; allows direct binary/multi-class judgment
Mitigation methods:
- Data-related: build faithful datasets with cleaner annotations; clean data automatically by filtering hallucinated examples; augment inputs with external knowledge
- Modeling and inference: modify encoder/decoder architecture for stronger semantic alignment; apply reinforcement learning with faithfulness rewards; implement controllable generation via planning and constraint satisfaction; deploy post-processing correction
Results
Key findings across task-specific analyses:
Abstractive Summarization — hallucination is endemic; prior work (Maynez et al. 2020) found 70%+ of generated summaries contain unsupported content, with 90% extrinsic and factually incorrect. ROUGE metrics correlate weakly with human faithfulness judgments.
Dialogue Generation — hallucination is prevalent; models generate plausible-sounding but unreliable facts especially in open-domain systems where ground-truth factual grounding is absent by design.
Generative QA — at an early stage; limited existing metrics and mitigation strategies; hallucination severity depends on whether the QA model has access to evidence.
Data-to-Text Generation — hallucination means contradicting the input table/structured data; relatively easier to detect and mitigate than open-domain tasks since source data is fully specified.
Machine Translation — hallucination includes under-translation, over-translation, and content divergence; translation-specific metrics exist but hallucination-specific evaluation remains limited.
Image Captioning and Vision-Language Tasks — object hallucination (inventing objects not in the image) is documented; severity varies with model architecture and training data quality.
Large Language Models (LLMs) — hallucinations are especially concerning as LLMs are deployed in high-stakes applications (medical QA, legal assistance); models like GPT-3 exhibit both intrinsic (misrepresenting training data) and extrinsic (inventing plausible-sounding facts) hallucinations; standard fine-tuning increases hallucination risk without appropriate mitigation.
Notes
Strengths: This is the first comprehensive, cross-task survey of hallucination in NLG. It unifies terminology across fragmented literatures (CV calls it "object hallucination"; NLP calls it "faithfulness" or "consistency"), systematizes root causes, and provides a roadmap of metrics and mitigation strategies. The coverage is broad—eight major NLG tasks—and the depth on each is sufficient to inform researchers entering those domains.
Weaknesses: Some sections are descriptive catalogs rather than comparative analysis (e.g., the metrics section reviews many approaches without systematic comparison of performance or trade-offs). The survey was published in 2022; hallucination in LLMs and generative AI has accelerated dramatically since, especially with GPT-3.5/GPT-4 and other large models. A 2024 update would reflect significant new developments in LLM hallucination measurement and mitigation. Task-specific sections vary in depth—summarization and LLMs are more developed; generative QA section is notably thin because research was nascent at publication time.
Future work: The paper identifies open research challenges: fine-grained hallucination sub-type detection (not just presence/absence); automatic fact-checking for extrinsic hallucinations; cross-task and cross-lingual metric generalization; tighter alignment between automatic and human evaluation; and how human cognitive models of hallucination should inform metric design.