Addressing climate change skepticism and inaction using human-AI dialogues

Contributions

• Proposes a novel approach to climate communication: AI-personalized dialogue tailored to individuals' specific climate change reservations rather than delivering universal messages.
• Demonstrates that LLM-driven conversations are substantially more effective than consensus messaging (the current standard-of-care) at reducing conviction in climate skepticism, even among Republicans.
• Shows strong effects on behavioral outcomes: willingness to make personal sacrifices, donations to climate charities, and intentions to sign pro-climate petitions.
• Provides mechanistic evidence that effects on general climate beliefs are mediated by reducing belief in the specific reservation targeted in the dialogue.
• Develops and releases DebunkBot, a public-facing demonstration tool for exploring the approach.

Method

Study 1 (N = 2,402 US residents, quota-sampled for gender and party):

Participants first articulated their main climate change reservation and rated their agreement (0–100 scale). They then engaged in a back-and-forth dialogue with GPT-4-turbo in one of three conditions:

1. LLM treatment: The model received the participant's stated reservation and engaged in a live conversation, providing factual information and addressing the specific concern.
2. Consensus messaging: Participants received a static, pre-written sentence on the overwhelming scientific consensus around human-caused climate change.
3. Control: Participants discussed an irrelevant, non-climate topic (iPhone vs. Android).

Participants completed 3–10 exchanges with the LLM. Belief measures were collected immediately post-conversation and at a 4-week follow-up.

Study 2 (N = 1,510 US residents):

Replicated Study 1 with additional behavioral measures: incentivized donation choice (how to split a $0.33 bonus between themselves and the Clean Air Task Force), willingness to sign pro-climate petitions, and actual petition signing.

Results

Specific climate reservations (primary outcome):

• LLM treatment reduced belief in the specific climate reservation by approximately 9 points (on a 0–100 scale; b = −9.03, 95% CI [−10.71, −7.33]) compared to control.
• This substantially exceeded the consensus messaging condition (b = −1.04, p < 0.001), with LLM treatment 1.7× more effective than consensus messaging.
• Effects held across all reservation types and were particularly strong for policy-related reservations and concerns about economic impacts.
• Party moderation: Effects were significant even for Republicans (b = −5.50, p < 0.001), demonstrating de-polarization.

General climate beliefs:

• LLM treatment shifted the general climate beliefs index by 2.2 points in a pro-climate direction (vs. 1.3 points for consensus messaging; p = 0.040).
• Largest effects on willingness to make personal sacrifices (b = 50–60 for human-causation items).
• Consensus messaging showed no significant effect on general beliefs compared to control (p = 0.886).

Behavioral outcomes (Study 2):

• Donations: Participants in the LLM treatment donated 15% more to the Clean Air Task Force (b = 1.35, p < 0.001) compared to control. Consensus messaging showed no significant increase (p = 0.146).
• Petition intentions: LLM treatment increased willingness to sign pro-climate petitions (b = 3.24, p = 0.001) vs. consensus messaging.
• Actual petition signing: No significant difference between LLM and consensus messaging on the binary choice to sign (p = 0.583).

Persistence:

• At 4-week follow-up, approximately 35–40% of the immediate treatment effect on specific reservations persisted.
• Relationship between post-treatment and re-contact beliefs remained significant for both LLM vs. control (b = 0.43, p < 0.001) and LLM vs. consensus (b = 0.42, p < 0.001).

Communication strategies:

Post-hoc analysis identified nine strategies the LLM employed (facts, co-benefits, relevance, norms, motivation, positive emotion, negative emotion, values, ingroup sources). Fact-based and co-benefit strategies were most common. Using facts and positive emotion predicted stronger belief reductions, while appeals to values and ingroup sources backfired (negative effects on specific reservations).

Notes

Strengths: Large, well-powered experiments with US nationally representative samples. Pre-registered analyses reduce p-hacking risk. Behavioral outcomes (donations, petitions) go beyond belief measures. Party-conditional analyses show effects across the political spectrum. The released DebunkBot tool enables public engagement with the approach.

Limitations: Sample is US-only and may not generalize to other national/cultural contexts. The study is a preprint and has not yet undergone peer review. Participants self-selected to engage in dialogue, which may limit real-world applicability—the authors note that only ~50% of the overall sample chose to interact with the bot. The binary petition-signing outcome lacked statistical power. LLM accuracy was manually verified but veracity assurance at scale remains an open challenge for deployment.

Open questions: Does the effect persist beyond one month? How do results vary with non-custom-prompted LLMs (e.g., ChatGPT in default mode)? What is the cost-benefit analysis for large-scale deployment considering LLM energy footprint? Can the approach work for more entrenched skeptics, or for audiences who won't voluntarily engage with AI?