Positional Bias

Positional bias in language models refers to systematic performance differences depending on where information appears in the input sequence. This includes primacy bias (preference for information at the beginning) and recency bias (preference for information at the end). The phenomenon is related to the serial-position effect observed in human cognition and memory.

Key papers

• [[2023-liu-lost-in-middle]] — reveals a U-shaped performance curve showing language models excel when relevant information is at the beginning or end, but significantly degrade when information is in the middle