Skip to content

Question Answering Systems

Question Answering (QA) systems automatically retrieve relevant information and generate answers to natural language questions. QA spans multiple paradigms:

Closed-domain QA: Answers drawn from a limited, curated knowledge source (e.g., a company's FAQ or medical database).

Open-domain QA (ODQA): Answers retrieved from large, heterogeneous corpora (Wikipedia, web) without pre-specified domains. Typically uses a two-stage architecture: dense or sparse retrieval (find relevant documents) followed by reading comprehension (extract or synthesize answer from document).

Machine reading comprehension (MRC): Models read a given passage and extract or generate an answer to a question about that passage. Foundational for the reader component of ODQA systems.

Conversational QA: Multi-turn interactions where context from prior exchanges informs interpretation of new questions.

Vulnerabilities

QA systems are vulnerable to misinformation injection at multiple stages: - Retrieval poisoning: False documents ranked highly by retriever - Adversarial reading: Models misled by adversarially-crafted passages - Hallucination: Generative readers produce plausible-sounding false answers

Key papers in this wiki

  • [[2023-liu-lost-in-middle]] — Demonstrates that language models struggle to retrieve and use information from the middle of multi-document QA contexts, showing a U-shaped performance curve and fundamental limitations in how transformers use long input sequences
  • [[2020-guu-realm]] — Pre-trains a language model jointly with a neural knowledge retriever using masked language modeling; outperforms prior ODQA systems by 4–16% on NaturalQuestions, WebQuestions, and CuratedTrec benchmarks
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Foundational work on retrieval-augmented generation (RAG) for knowledge-intensive QA
  • Survey of Hallucination in Natural Language Generation — Survey Section 9 covers hallucination definition, metrics, and mitigation in generative QA systems
  • On the Risk of Misinformation Pollution with Large Language Models — Analyzes vulnerability of ODQA systems to LLM-generated misinformation; proposes defenses