Faithfulness refers to the consistency between generated output and its source or input. A faithful summary accurately reflects the source document; a faithful answer is grounded in retrieved evidence; a faithful translation preserves meaning from the source language. Faithfulness is fundamental to trustworthiness in conditional text generation tasks.
Faithfulness is distinct from factuality. A summary can be faithful (consistent with its source) but factually incorrect if the source itself contains errors. Conversely, a summary could be unfaithful (inconsistent with its source) but factually correct if it happens to state true information not in the source. However, in practice, unfaithful text often violates factuality as well.
Measurement and evaluation
Reference-based metrics:
- Natural Language Inference (NLI): Does the output entail the source?
- Question-Answering: Generate QA pairs from output; can they be answered from the source?
- Semantic similarity: Do embedding-based similarities align between output and source?
Reference-free approaches:
- Textual entailment classifiers trained on consistency judgments
- Dependency-based consistency checking
- Information extraction overlap (comparing entities and relations)
Human evaluation:
- Annotators assess whether generated text is supported by source material
- Attribution scoring: identifying which claims are grounded where
- Faithfulness annotation on datasets like XSum, QAGS, QuestEval
Key papers in this wiki
- Maynez et al. (2020) — Large-scale study quantifying faithfulness failures in abstractive summarization; introduces the intrinsic vs. extrinsic hallucination taxonomy; shows ROUGE metrics correlate weakly with faithfulness; proposes entailment-based automatic evaluation.
Open questions
-
Automatic evaluation scalability: Can reference-free methods reliably measure faithfulness without human annotation?
-
Domain generalization: Do faithfulness benchmarks built in one domain (summarization) transfer to others (dialogue, machine translation, QA)?
-
Training for faithfulness: How can models be trained to prioritize faithfulness without sacrificing fluency or informativeness?
-
Interpretability: When models produce unfaithful output, can we identify which components (architecture, training, decoding) are responsible?
Related topics
- factuality — whether output is true in the world
- hallucination — unfaithful and false output
- abstractive-summarization — task prone to faithfulness failures
- natural-language-generation — broader task area
- text-generation-evaluation — metrics for measuring output quality
