Reducing belief in conspiracy theories as they unfold using large language models

Contributions

• Demonstrates that brief LLM-based conversations can reduce conspiracy belief in real-time, immediately after crisis events when conspiracy narratives are forming
• Identifies seven persuasion strategies employed by the LLM across two distinct conspiracy topics (political assassinations)
• Provides evidence for downstream effects on subsequent conspiracy beliefs, suggesting that initial debunking may shape how people evaluate related misinformation
• Advances the study of AI-mediated interventions for combating misinformation at scale

Method

Two pre-registered experiments conducted within days of two assassination events: the July 2024 Trump assassination attempt and the September 2025 Charlie Kirk assassination. U.S. adult participants (Experiment 1: $N = 472$; Experiment 2: $N = 1042$) were randomly assigned to one of three conditions:

1. Debunking Dialogue: Multi-turn conversation with an LLM designed to reduce conspiracy belief through evidence-based debunking
2. Information List: Participants viewed a static list of factual information about the event
3. Irrelevant Dialogue: Control condition discussing an unrelated topic with the LLM

Participants completed open-ended and closed-ended measures of conspiracy belief immediately after the intervention and again after approximately one to two months. The study also coded LLM-generated responses to identify the rhetorical strategies the model employed.

Results

Main effects: The LLM debunking dialogue significantly reduced conspiracy belief about the assassination attempt compared to both control conditions (Trump: $b = -6.95$, $p < .001$; Kirk: $b = -7.48$, $p < .001$). The same pattern held for beliefs about cover-ups, hidden factors, and evaluations of the event's authenticity. No effect was found on trust in official explanations for the Trump case, possibly because no clear official account had been published at the time of data collection; the Kirk debunking did increase trust in official explanations.

Rhetorical strategies: The LLM employed seven primary strategies:

• Logic, facts, and evidence (most frequent for emerging events with little public information)
• Stories, metaphors, and examples
• Building rapport and validating concerns
• Emphasizing unknowns and epistemic uncertainty
• Highlighting the harms of conspiracy thinking
• Socratic questioning
• Directing attention to credible sources

Notably, the model's approach differed between the two events: for Trump (where information was sparse), the LLM relied more on epistemically cautious framing and open questions; for Kirk (where more information was available), it deployed more direct factual rebuttals.

Downstream effects: Participants who received the debunking dialogue showed reduced belief in other, unrelated conspiracy theories approximately one to two months later, with some evidence of indirect mediation through immediate belief change. However, evidence for persistence effects was mixed: some specific conspiracy beliefs declined but others did not, suggesting the intervention's scope was neither unlimited nor fully negligible.

Notes

The timing of these experiments—within days of actual crisis events—is a significant methodological strength, capturing conspiracy belief formation in real-time rather than studying settled, historical narratives. The work addresses an important gap in the intervention literature: most prior work has focused on debunking well-established conspiracy theories; this paper tests whether AI can counter beliefs as they form, when misinformation is most malleable.

The identification of distinct rhetorical strategies across events is instructive: the model adapted its approach to the information landscape, prioritizing epistemic caution when facts were scarce and evidential reasoning when information became available. This suggests that AI dialogue systems may have inherent advantages over static fact-checks in real-time crisis settings.

The carry-over effects on subsequent conspiracy beliefs are suggestive but not conclusive, warranting replication. The absence of effects on trust in official explanations for the Trump case is noteworthy and may reflect the lack of consensus about the event's details at the time of the study.