On Faithfulness and Factuality in Abstractive Summarization

Contributions

• Large-scale hallucination analysis: First comprehensive human evaluation quantifying hallucination prevalence across multiple neural architectures (RNN, CNN, Transformer-based systems) on the XSum extreme summarization dataset, revealing that >70% of summaries contain hallucinations.
• Hallucination taxonomy: Formal distinction between intrinsic hallucinations (misrepresenting source content) and extrinsic hallucinations (introducing facts unsupported by the source); empirical finding that 90% of hallucinations are extrinsic and erroneous.
• Evaluation of automatic metrics: Demonstrates that ROUGE and BERTScore correlate weakly with human judgments of faithfulness and factuality; shows that textual entailment and question-answering measures provide better automatic proxies for semantic fidelity than surface-level metrics.
• Pretraining effects: Evidence that models initialized with pretrained language model parameters (BERT, GPT-2) show improved faithfulness compared to randomly initialized baselines, suggesting that access to world knowledge reduces hallucination.

Method

The authors conduct a controlled study on the XSum extreme summarization dataset (226,711 BBC articles with single-sentence gold summaries). They evaluate summaries from four neural abstractive systems:

• PtGen: Pointer-Generator network (See et al., 2017) with copy mechanism.
• TConvS2S: Topic-aware Convolutional Sequence-to-Sequence (Narayan et al., 2018a).
• TransS2S: Transformer Seq2Seq baseline.
• BertS2S: BERT-initialized Encoder-Decoder with shared Transformer layers.

Hallucination Detection: A random sample of 500 articles is manually annotated by three fluent English annotators. For each generated summary, annotators identify hallucinated spans and classify them as intrinsic or extrinsic. Annotators are instructed to: (i) identify any text in the summary that is unsupported by or contradicts the source document, and (ii) classify whether the hallucination involves source information misrepresented (intrinsic) or invented entirely (extrinsic). Inter-annotator agreement (Cohen's κ) is reported.

Automatic Evaluation: The authors train an entailment classifier (BERT-based on Multi-NLI) to score whether a summary entails the source document, and implement question-answering-based evaluation (generating questions from summaries and checking if they're answerable from the source using BERT reading comprehension).

Results

Hallucination Prevalence:

• PtGen: 63.3% of summaries contain ≥1 hallucination; 24.7% contain factual hallucinations.
• TConvS2S: 71.5% hallucinated; 21.5% factual.
• TransS2S: 68.1% hallucinated; 20.7% factual.
• BertS2S: 64.1% hallucinated; 26.9% factual.

Intrinsic vs. Extrinsic:

• Overall, extrinsic hallucinations comprise ~90% of all hallucinations.
• Gold summaries from the XSum dataset contain 7.4% extrinsic hallucinations, suggesting dataset artifacts (journalist-written summaries sometimes use inference beyond the article).
• Models show higher extrinsic hallucination rates (19.9–27.3%), indicating fundamental model vulnerabilities rather than dataset bias alone.

Faithfulness Metrics:

• BertS2S achieves the highest faithfulness (34.7% faithful and/or factual) despite having only average ROUGE scores.
• ROUGE and BERTScore correlate weakly with human-annotated faithfulness (Spearman's ρ ≈ 0.2–0.3), while textual entailment scores correlate better (ρ ≈ 0.4–0.5).
• Question-answering-based evaluation (checking if QA pairs generated from summaries are answerable from the source) shows moderate correlation (ρ ≈ 0.4) with human factuality judgments.

Pretraining Helps:

• 34.7% of BertS2S abstracts are marked faithful and/or factual, compared to 26.9% for PtGen—a 7.8% absolute improvement.
• Pretraining-initialized models have the lowest extrinsic hallucination rates (accounting for the fact that extrinsic hallucinations are both intrinsically hallucinated and factually wrong).

Notes

Strengths:

• Addresses a real and widespread problem: existing metrics (ROUGE, BERTScore) reward fluent but unfaithful summaries, incentivizing models to hallucinate.
• Carefully designed annotation schema distinguishing intrinsic vs. extrinsic hallucinations captures the source of model errors: models don't just paraphrase poorly, they invent.
• Large-scale evaluation (500 articles × 4 systems × 3 annotators) provides robust empirical evidence; inter-annotator agreement for factuality classification is reported.
• Clear practical implications: highlights that pretraining and semantic-inference-based metrics offer better pathways to faithful summarization than purely training-objective changes.
• Influential methodology: the hallucination taxonomy and human evaluation approach have become standard in subsequent work on LLM hallucination.

Weaknesses:

• Limited to extreme summarization (single-sentence): findings may not transfer to longer, multi-sentence abstractive summaries (news articles, scientific papers), where intrinsic hallucinations might be more frequent.
• Evaluation limited to English and one dataset (XSum): generalization to other languages, domains (scientific, clinical), and summary lengths unknown.
• QA-based automatic evaluation itself requires a trained QA system and is therefore not a ground-truth measure; circular dependency if the QA model also hallucinates.
• No exploration of how hallucinations harm downstream users: Do readers notice? Do they trust hallucinated summaries? Impact on real-world usage unclear.
• Modest insights into why hallucinations occur: the paper documents frequency but doesn't deeply analyze failure modes or identify which source characteristics lead models to hallucinate.

Future directions:

• Training objectives that penalize hallucinations (e.g., reinforcement learning with faithfulness rewards).
• Decoding strategies (e.g., constrained generation, beam search with entailment filtering) that reduce hallucination at inference time.
• Cross-domain evaluation: do these findings hold for news, scientific papers, medical abstracts?
• Understanding user perception: do people notice and trust hallucinated summaries in real-world applications?
• Integration with knowledge bases: can knowledge-grounded decoding (retrieving facts from external sources) reduce extrinsic hallucinations?