Skip to content

Source Credibility

Source credibility refers to the perceived trustworthiness and competence of a source of information. In the context of misinformation and disinformation, source credibility is both a psychological mechanism (how humans evaluate sources) and a design principle (how platforms and fact-checking systems ought to assess information sources).

Key dimensions

Expertise: Does the source have specialized knowledge relevant to the claim? Medical claims from physicians vs. laypeople; scientific claims from peer-reviewed sources vs. social media.

Trustworthiness: Is the source motivated to tell the truth, or do they have incentives to deceive? News outlets with editorial standards vs. agenda-driven blogs; institutional sources vs. partisan actors.

Track record: Does the source have a history of accurate or inaccurate claims? LLMs can condition on a poster's prior accuracy (prior fact-check history), and humans weight repeat offenders differently.

Bias and conflicts of interest: Does the source have apparent conflicts of interest or ideological alignment that might color their reporting?

Institutional affiliation: Is the source affiliated with reputable institutions (universities, major news organizations) or fringe outlets?

Role in misinformation

Misinformation often succeeds by exploiting source credibility heuristics: claiming false credentials, impersonating trusted sources, or leveraging social proof from high-follower-count accounts. Conversely, fact-checkers rely heavily on source credibility to assess evidence reliability (e.g., prioritizing peer-reviewed studies over opinion blogs when verifying health claims).

LLM-based fact-checkers can exhibit speaker bias—weighting the speaker's occupational status, nationality, or party affiliation in ways that distort truth judgments independent of content quality. This represents a failure of source credibility assessment: the system is over-weighting irrelevant attributes (status stereotypes) while potentially under-weighting relevant cues (expertise, track record).

Key papers

  • From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe — Demonstrates that source-credibility assessment in LLMs can be reversed by post-training: the Tülu-3 recipe teaches models to distrust lower-status sources, reversing the base model's slight leniency. Shows that source-credibility signals are malleable and recipe-specific.
  • Wack & Warren (2026) — Analyzes how Grok, Community Notes, and professional fact-checkers' verdicts vary by whether the queried post is true or false and by topic, revealing asymmetries in which sources receive correction notes.
  • Speaker Bias — Specific mechanism by which source identity distorts credibility judgments.
  • Fact-checking systems — Systems that must integrate source-credibility assessment with evidence-based verdict generation.
  • Trust in institutions and communicators — Psychological and institutional foundations of credibility judgments.
  • Media bias — Systematic credibility variations across media outlets and their political or institutional affiliations.
  • Fake news — Misinformation often succeeds by manipulating source-credibility signals or impersonating trusted sources.

Methodological challenges

  • Confounding content with source: Hard to isolate source effects from content effects; minimal-pair and counterfactual study designs (varying only source identity) are essential but labor-intensive.
  • Measuring what systems actually use: LLMs can encode source credibility in ways that aren't obvious (residual-stream directions, subtle probability shifts); mechanistic interpretability is required to understand how source information flows through the system.
  • Generalizability across domains: Source-credibility heuristics vary by topic (medical, political, scientific); a model trained on one domain may apply mismatched heuristics to another.