Durably reducing conspiracy beliefs through dialogues with AI

Key findings

• Primary effect: AI-guided dialogue reducing conspiracy belief by ~20 points (Cohen's d = 1.1); replicates across multiple belief domains and belief-strength levels.
• Durability: No decay over 2 months; effect persists even accounting for ~15% participant attrition.
• Generality: Works across diverse conspiracies (conspiracies about JFK, 9/11, COVID, 2020 election, aliens, etc.); effect size does not vary significantly by conspiracy type.
• Transfer: ~8% reduction in belief in other (unaddressed) conspiracy theories, suggesting belief in the AI-corrected conspiracy makes people reconsider their broader conspiratorial worldview.
• Mechanisms study: Facts and evidence are the load-bearing mechanism; rapport-building, common-ground framing, narrative/metaphor, and explicit persuasive intent all negligible or absent. Telling the AI "just correct factual errors, don't try to persuade" works as well as explicit persuasion instructions.
• Real-time conspiracies: When debunking active, unfolding conspiracies (tested on Trump assassination attempt, 2024), the effect persists but operates via epistemic humility ("we don't know yet") rather than refuting false evidence.
• Accuracy: A fact-checker rated AI statements: 99% true, 1% misleading, 0% false, 0% politically biased.
• Behavioral downstream effects: Increased inclination to mute/disconnect from conspiracy-spreading accounts; increased likelihood to argue against (rather than agree with) conspiracy claims in conversation; increased trust in AI.

Method and design

Main study (~1,000 U.S. participants, stratified for conspiracy belief):

1. Elicitation: Participants write in their own words a conspiracy they believe and cite evidence for it (open-ended, not multiple-choice).
2. Baseline belief: Rate belief 0–100. Only retain participants above 50.
3. Intervention (randomized): - Treatment: Three-round dialogue with GPT-4 Turbo (given the participant's specific belief and evidence; instructed to persuade them it's false). - Control: Three-round dialogue about an unrelated topic (dogs vs. cats, firefighters, etc.).
4. Post-test: Re-rate belief on the same 0–100 scale; follow-ups at 10 days and 2 months.

Mechanism follow-up (~1,300 participants):

Replicate the baseline effect, then test competing explanations via factorial variation:

• Perceived bias of AI: Tell some participants the AI will try to persuade them (vs. neutral framing). No difference.
• Dialogue framing: Reframe as a "debate" where the human should try to convince the AI. Still works.
• Information overload: Instruct the AI to be concise (cutting response length in half). No difference; if anything, works slightly better.
• Persuasive intent: Instruct the AI to "just correct factual errors" (remove persuasion instruction). Works equally well.
• No evidence condition: Instruct the AI to persuade but without using facts, evidence, expert consensus, or counterarguments. Does not work—this condition eliminates the effect, proving facts are essential.
• Human explanations: Using text-analysis of open-ended responses, participants who shifted their beliefs cited "facts and logic" (majority); non-shifters cited evidence having insufficient novelty/depth or not understanding the AI's disagreement.
• Strategy coding: Code each AI response for debunking strategies (rapport, offering alternative explanations, pointing out inconsistencies, citing expert consensus, stories/metaphors, etc.). Correlate each strategy's prevalence in a conversation against belief reduction. Result: facts/evidence and sources/experts predict reduction; rapport and narratives do not.

Connected work

Within the fake-news/misinformation literature:

• Related to backfire-effect (earlier belief in misinformation's resistance to correction) — Rand's work shows correction can work at scale if done right.
• Related to rumor debunking and fact checking interventions — provides a novel intervention pathway (AI dialogue vs. static fact-checks).
• Related to conspiracy belief psychology — challenges the "motivated reasoning" narrative that once someone goes "down the rabbit hole" they're unreachable; argues that insufficient access to personalized counter-evidence was the bottleneck, not motivational imperviousness.
• Related to psychological interventions misinformation — offers a behavioral/conversational intervention; comparable in scope to inoculation and pre-bunking work but operates on existing false beliefs.

In AI/NLP applications:

• Demonstrates the persuasive capacity of LLMs, echoing concerns about llms misinformation risk but points to positive-use case.
• Related to work on dialogue systems and interactive ai as an experimental tool in social science (the "Qualrix + LLM" paradigm Rand highlights).

Implications and limitations

Strengths:

• Large, diverse samples (N ~1,000–1,300) with high-quality measurement (0–100 scale, follow-ups, replication).
• Clear experimental manipulation and mechanism dissection: the follow-up study is notably rigorous in ruling out confounds (bias, framing, overload, persuasive intent).
• Practical generalizability: effects hold across open-ended conspiracy beliefs, not a single curated target; behavior shift measured alongside belief shift.
• Transparency and reproducibility: Rand provides the Qualrix survey templates publicly; cites the preprint.

Limitations and open questions:

• Generality to real-world populations: Experiments use paid survey respondents (Qualtrics panels), though Rand notes debunkbot.com data (~100K visitors) shows comparable or better effects in unpaid public samples—encouraging but not a complete solution.
• Scalability and deployment: Real-world deployment (Reddit, Twitter, Google ads targeting conspiracy keywords) is described as future work, not yet evaluated. Raises questions about: How to reach people before they've committed to a belief? How does the effect interact with social network reinforcement of conspiracy beliefs?
• Safety and misuse: The talk acknowledges LLMs' capacity to spread misinformation using the same dialogue machinery. Rand notes ongoing work comparing persuasion-to-truth vs. persuasion-to-falsehood, but results are not yet public.
• True conspiracies: ~1% of open-ended beliefs were actual conspiracies (e.g., historical plots with evidence); the AI did not reduce belief in these, and if anything slightly increased belief. This is correct behavior, but the paper does not deeply probe the boundary (e.g., what about recently exposed conspiracies where counter-evidence is incomplete?).
• Transfer of belief change: The 8% spillover to other conspiracies is encouraging but modest. Suggests the mechanism is "I was wrong about this one, so maybe I'm wrong about others" rather than a deep shift in epistemology or conspiracy-proneness.

Notes

This work is important to the misinformation field for three reasons:

1. Empirically challenges a widespread narrative: The idea that conspiracy beliefs are effectively immune to correction once formed is deeply embedded in psychology and communication research (e.g., the "backfire effect"). Rand's experiments, properly controlled, show that personalized evidence can reach people. This doesn't mean all false beliefs are equally movable, but it shifts the burden of proof.

2. Mechanism clarity: Most prior work on belief correction (fact-checks, inoculation, etc.) remains somewhat "black-box"—we know certain interventions work, but the why is unclear. The mechanism study here is unusual in the field for its rigor: testing competing hypotheses in parallel, using strategy coding, and soliciting participant explanations. The conclusion—facts are load-bearing, rapport is incidental—is valuable guidance for future intervention design.

3. LLM as an experimental tool: Rand's framing of the LLM as a controllable "tutor" (not just a subject of study) is interesting for empirical social science more broadly. The Qualrix + GPT integration he highlights allows for a new class of personalized, adaptive experiments.

Caveats:

• The follow-up mechanism study is described as "about to be posted as a working paper"; the video was recorded in early 2025, so publication and peer review status should be verified.
• Rand explicitly positions this as "we were wrong to think facts don't work" — a position that may overcorrect against earlier pessimism about conspiracy belief durability. The actual picture is likely more nuanced (facts work better for some beliefs, some populations, some framing contexts than others), but that nuance is not resolved here.
• The downstream behavioral measures (muting accounts, arguing rather than agreeing) are intent-to-behave, not actual behavior; real-world follow-up is absent.

See also

• conspiracy-theories — belief in false conspiracies; psychological drivers and interventions
• misinformation-correction — corrections, fact-checks, debunking, and other post-exposure interventions
• llm — large language models in misinformation research; detection, generation, and persuasion
• van Boekel et al. on dialogue-based belief change — related work on conversation-based interventions (not yet in wiki)
• Belief-change interventions — inoculation, prebunking, and psychology-informed approaches to shifting attitudes