Skip to content
Fact-check? True!: Tracking Verification Requests and the Absence of Truth Verdicts across Correction Systems

Fact-check? True!: Tracking Verification Requests and the Absence of Truth Verdicts across Correction Systems

Authors: Morgan Wack, Patrick Warren

Venue: arXiv cs.SI, October 2026 — arXiv:2610.17068

TL;DR

Examining 47,899 X/Twitter users asking Grok (X's AI assistant) whether posts are true, the authors find that ~51% of questioned posts are actually accurate, yet correction systems (Community Notes, professional fact-checkers) focus overwhelmingly on false claims. Professional fact-checkers publish true verdicts in only 8–11% of their output; Community Notes contributors write verifying notes on just 27.3% of questioned posts, almost none of which are displayed. This reveals a structural asymmetry: platforms' correction infrastructure targets false claims while largely ignoring requests to confirm accurate information.

Contributions

  • Dataset of verification queries: 47,899 Grok queries spanning January–February 2026, linked to Community Notes data, revealing what users publicly doubt on social media.
  • Asymmetry in content accuracy: Users question content at roughly equal rates regardless of truth; 51.4% of queried posts are true, contradicting the assumption that most user-flagged content is false.
  • System-level comparison: Quantifies how Grok (responsive to individual queries), Community Notes (contributor-selected), and professional fact-checkers (organizational selection) differ in output composition. Grok addresses 51.4% true verdicts; Community Notes writers favor flagging false claims (7% of true posts get displayed notes vs. 18.7% of false posts).
  • Absence of confirmation verdicts: Professional fact-checkers rarely publish true verdicts (8–11% across 39 organizations); Community Notes excludes notes verifying posts from public display by design.

Method

  • Primary coding: 100 Grok responses manually coded by two human annotators (agreement κ=0.76–0.97 across verdict type and dispute type) using Claude-haiku-4-5 LLM for structured coding with human validation.
  • Secondary verification: Perplexity's search-grounded model evaluated 600 queried posts to check whether Grok verdicts are artifacts of a confirmatory model; found Grok's distribution of true/false verdicts differs from LLM defaults (84.2% vs. 79.8% agreement on true verdicts).
  • Community Notes alignment: Joined 36,675 queried posts to public Community Notes data (October 2026 release, 3.1M notes since January 2021). Tracked whether contributors wrote verifying vs. flagging notes and whether notes were displayed (subject to X's rating filters).
  • Public expectations: Verasight omnibus survey of 1,000 U.S. adults asking respondents' expectation of Grok's true-verdict rate; median estimate 45–54%, matching observed distribution.

Results

Rates of doubted content accuracy (Figure 2): - Grok rendered true/false verdicts on 19,042 queried posts: 51.4% true, 48.6% false. - Survey respondents expected roughly balanced true/false verdicts (45–54% true), contradicting the premise that systems should focus on false claims.

Community Notes vs. Grok (Section 5.2, Figure 3): - Of 29,294 top-level posts queried, 24,729 (84.4%) received no Community Note at all. - 4,565 (15.6%) received any note; only 968 (3.3%) had a displayed note. - Among queried posts Grok judged true, 89.6% received no note; Grok-judged false posts saw notes 4× more often (7.0% vs. 1.6%). - Contributors wrote 2,981 notes verifying posts; 27.3% of all notes on queried posts were verifications, yet zero were displayed due to X's scoring filter.

True verdicts across systems (Table 2, Figure 5): - Grok: 51.4% true on answered queries. - Community Notes: 27.3% of written notes verify posts; over time (2015–2024), true-verdict share fell from ~13% to ~5–7%. - Professional fact-checkers (39 organizations, FACTors dataset): 8.0% true verdicts; Wisconsin Watch outlier at 73.7%, indicating structural rather than accuracy-driven variation.

Accuracy and notes by topic (Table 3): - Truth share ranges 49.4% (religion/identity) to 58.3% (science/climate). - Notes received vary 1.9% (sports) to 5.1% (technology); within every topic, inaccurate posts received displayed notes more often than accurate posts.

Connections

  • Fact-checking and corrections — documents the gap between user verification requests and fact-checker response patterns.
  • Community notes — empirical analysis of Community Notes' design choice to exclude verifying notes from public display and its consequence for addressing "incorrect rejection" (users doubting accurate claims).
  • Content moderation — reveals asymmetry in platform moderation logic: systems default to flagging false claims, leaving true claims unconfirmed even when publicly doubted.
  • Crowdsourced governance — examines whether crowd-sourced moderation scales to handle both refutation and confirmation tasks equally.
  • Verification — user-initiated verification requests as a signal of information uncertainty independent of claim accuracy.

Notes

The paper's core insight—that platforms' correction infrastructure was designed with false-claim refutation in mind, not true-claim confirmation—has implications for system design. The finding that 51% of questioned posts are accurate suggests verification-seeking behavior does not perfectly track falsity. The exclusion of Community Notes verifying posts from public display appears deliberate, coded into X's scoring logic, yet no public documentation explains why confirmations should be hidden while refutations are surfaced. The authors discuss whether displaying verifying notes would improve user accuracy beliefs without encouraging overconfidence, positioning this as a design choice rather than an inevitable technical limitation.

Methodologically, the coding process (LLM-assisted with human review) reflects contemporary practice in large-scale content analysis; the intercoder agreement metrics (κ=0.76–0.97) are strong, and the secondary model agreement check (Perplexity) adds robustness against systematic bias in the coding process. The survey evidence of public expectations (45–54% true verdicts) is limited by sample size and U.S.-only scope but provides a reality check on whether the system's asymmetry reflects user demand or platform choice.