Fact-checking information from large language models can decrease headline discernment

Contributions

• Preregistered randomized controlled experiment (N=2,159) comparing LLM-generated fact checks (ChatGPT 3.5) to human fact checks and control conditions
• Demonstrates a critical gap between LLM accuracy and human behavioral outcomes in fact-checking interventions
• Identifies specific harmful effects contingent on LLM judgment accuracy: false negatives (mislabeled true headlines) and uncertainty about false headlines
• Shows strong selection bias in opt-in scenarios: participants choosing to view LLM fact checks are significantly more likely to share both true and false news

Method

The study recruited a representative U.S. sample and presented participants with 40 real political news headlines (half true, half false; balanced for partisanship). Participants were randomly assigned to four conditions:

1. Control: headlines only, no fact-checking information
2. Human fact check: traditional fact checks with clear verdicts, publisher reputation assessment, and supporting details
3. LLM-forced: ChatGPT-generated fact checks presented directly below the headline
4. LLM-optional: participants could click to reveal ChatGPT fact checks

Two separate groups (belief and sharing) rated either whether they believed headlines were accurate or whether they would share them. ChatGPT's fact-checking accuracy was independently coded as "True," "False," or "Unsure" by three authors (Krippendorff's α = 0.79). Analysis used Mann–Whitney U tests with Bonferroni correction for multiple comparisons.

Results

LLM Accuracy

ChatGPT correctly identified 90% of false headlines (18/20) but was less accurate on true headlines: only 15% correct (3/20), 20% mislabeled as false, and 65% marked as "unsure."

Main Effects: Ineffectiveness of LLM Intervention

Overall, LLM fact checks produced no significant improvement in headline discernment in either the forced or optional conditions:

• Belief group (forced): −4.50% discernment change (not significant)
• Sharing group (forced): −0.43% discernment change (not significant)

Human fact checks, by contrast, significantly improved discernment:

• Belief: +18.06% (p < 0.001)
• Sharing: +8.98% (p = 0.001)

Contingent Harmful Effects (When Accuracy is Considered)

When examining specific accuracy scenarios, the LLM caused measurable harm:

• True headlines mislabeled as false: −12.75% belief (p < 0.001)
• False headlines with LLM uncertainty: +9.12% belief (p = 0.03)

Similar mixed results appeared in the sharing group: +11.09% for correctly identified true headlines, but +9.77% for uncertain false headlines.

Opt-In Behavior

In the optional condition, participants showed strong selection bias. Those who chose to view LLM fact checks (>50% of headlines) were significantly more likely to:

• Share both true and false news (+29% to +39% across scenarios)
• Believe false headlines accurately identified by the LLM (+29.35%)
• Believe false headlines the LLM was unsure about (+28.12%)

Attitudes Toward AI and Partisan Effects

Participants with positive attitudes toward AI who viewed LLM fact checks were significantly more likely to share headlines across all scenarios. Partisan incongruence with true headlines that the LLM was unsure about significantly decreased belief and sharing intent, regardless of opt-in status.

Notes

This paper presents an important cautionary finding about deploying LLMs for scalable fact-checking: accuracy alone does not guarantee beneficial human outcomes. The "breaking news problem" (LLMs' difficulty with novel events) and the role of uncertainty expressions deserve further investigation. The study's strength lies in its preregistration, representative sample, and comparison of AI to human baselines. Limitations include use of a limited headline set, a single LLM model and prompt, and survey-based rather than real-world sharing behavior, though prior work correlates survey willingness with actual sharing. The findings underscore the need for policy and system design interventions before deploying AI-generated fact checks at scale.