Speaker Bias

Speaker bias refers to the phenomenon where the credibility or truth judgment of a claim varies based on attributes of the person making the claim—such as their gender, nationality, political party, occupational status, or other demographic characteristics—independent of the content of the claim itself. In misinformation contexts, this becomes particularly salient: do fact-checking systems (human or automated) evaluate claims differently depending on who says them?

Mechanisms and concepts

Testimonial injustice (Fricker, 2007): When credibility judgments are biased by the listener's prejudices about the speaker, leading to epistemic injustice—a distinctive form of wrongdoing in which someone is wronged in their capacity as a knower. In the context of misinformation, a fact-checking system that doubts claims from lower-status speakers commits testimonial injustice.

Attribute-based susceptibility: LLMs' truth judgments shift with speaker attributes in ways that typically correlate with social stereotypes. Prior work has documented effects of speaker nationality, gender, and political party affiliation on whether LLMs rate identical claims as true or false.

Recipe-specificity: Recent research (AlDahoul & Zaki, 2026) demonstrates that speaker bias in LLMs is not a general property of post-training but depends on the specific recipe used—same base model, different post-training procedures, can yield opposite biases—suggesting that bias is a learnable property of training data or objectives.

Key papers

• From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe — Shows that the Tülu-3 post-training recipe creates job-based doubt in OLMo-2 32B (lower-status posters' claims rated false more often), but this effect does not appear in Meta's, Alibaba's, or Mistral's post-training, and deployed commercial models show no such bias. Locates the mechanism to a single residual-stream direction that reverses during supervised fine-tuning.

Related concepts

• Source Credibility — Broader topic encompassing source reputation, track record, and affiliation effects on claim credibility.
• Social Bias — General category including gender, race, nationality, and status biases in ML systems.
• Testimonial injustice (Fricker, 2007) — Philosophical framework for understanding credibility disparities as forms of epistemic injustice.
• Fact-checking systems — Systems that assess claim truth and must navigate the trade-off between content-only and speaker-aware evaluation.
• Bias in Language Models — Broader study of how social biases persist in or are introduced by post-training.

Connections

• Social bias in LLMs — Speaker bias is one manifestation of social bias in LLMs; mechanistic studies reveal how bias is represented and transformed during training.
• Content moderation — Related to content-level moderation decisions that may vary by speaker identity.
• Polarization — Political speaker bias can amplify partisan divides if fact-checking systems treat claims differently by party.