Knowledge Retrieval

Knowledge retrieval methods augment language models with external information sources—documents, knowledge graphs, or the internet—retrieved dynamically during inference. Approaches range from simple concatenation of retrieved documents to multi-stage retrieval that iteratively refines context based on reasoning steps. This avoids modifying model parameters while enabling access to real-time information.

Key papers

• Zhang et al. (2023) — Survey of retrieval-augmented and internet-enhanced methods, covering single-stage and multi-stage retrieval trade-offs

Related topics

• Large Language Models
• Knowledge Editing
• Continual Learning
• Information Retrieval

Notes

Retrieval-based approaches scale well and require no model modification but introduce retrieval latency and potential noise. Effectiveness depends critically on retrieval quality; irrelevant or noisy documents can degrade generation. Internet-enhanced variants provide unlimited knowledge but suffer from high inference overhead and noisy web content.