On the belief that beliefs should change according to evidence: Implications for conspiratorial, moral, paranormal, political, religious, and science beliefs

Contributions

• Introduces and validates the Actively Open-minded Thinking about Evidence (AOT-E) scale — an 8-item measure of the meta-belief that beliefs ought to change in response to evidence, distinct from general cognitive sophistication.
• Demonstrates across three large studies (N=375, 539, 778) that AOT-E strongly predicts skepticism of conspiratorial, paranormal, and religious claims; rejection of traditional moral values; and acceptance of science.
• Shows that AOT-E is far more predictive for political liberals (Democrats) than conservatives (Republicans), with the relationship frequently non-significant or even reversed for conservatives, suggesting political ideology moderates how evidence-responsive reasoning translates to actual beliefs.
• Reveals a dissociation between believing beliefs should change according to evidence and actual belief formation: higher AOT-E does not reduce polarization on highly ideological issues (e.g., capital punishment), implying that meta-beliefs about evidence operate differently across domains.

Method

Study 1 (Mechanical Turk, N=375): Administered the AOT-E scale (8 items asking whether beliefs should change with evidence, with reverse-scored items capturing identity protection and dogmatism) alongside measures of conspiratorial ideation, paranormal beliefs, traditional moral values, political ideology, free market ideology, pro-science beliefs, God skepticism, and demographics. Cronbach's α for AOT-E = .87. Measures converted to standardized 0–100 point scale (POMP).

Study 2 (Lucid for Academics, N=539): Used a revised AOT-E scale (changed "beliefs" to "opinions" for some items to avoid religious belief presumptions, Table 7) alongside conspiracy beliefs, paranormal beliefs, moral values (care/fairness, in-group loyalty, authority, sanctity/divinity, and liberty/proportionality), political ideology, pro-science beliefs (13 items), religious beliefs, political party preference, and the Cognitive Reflection Test (CRT; 3 items measuring analytic thinking). Cronbach's α for revised AOT-E = .74.

Study 3 (Lucid for Academics, N=778): Replicated Study 2 design with stratified sampling by political party to ensure adequate Democrat (N=302), Republican (N=223), and Independent (N=239) representation. Included the Cognitive Reflection Test and examined correlations separately by party.

Analytic approach: Pearson correlations between AOT-E and each belief/value domain; party-stratified analysis to test whether AOT-E's predictive validity varies by political ideology.

Results

Overall predictive pattern (Studies 1–3): AOT-E was strongly negatively correlated with:

• Conspiratorial beliefs (r = –.36 to –.20)
• Paranormal beliefs (r = –.40 to –.14)
• Religious beliefs (r = –.42 to –.52)
• Belief in God (r = –.52)
• Traditional moral values (r = –.54 to –.37)
• Political conservatism (r = –.46 to –.61)

And positively correlated with:

• Pro-science beliefs across domains (e.g., global warming r = .37–.55, evolution r = .51, vaccines r = .47)
• Care/fairness moral values (r = .26–.47)
• General trust in scientists (r = .35–.40)

Critical ideological moderation (Study 3): The relationship between AOT-E and beliefs was dramatically different for Democrats vs. Republicans:

• Global warming: Democrats (r = –.43), Republicans (r = .09, n.s.)
• Evolution: Democrats (r = .28), Republicans (r = <.01, n.s.)
• Capital punishment: Democrats (r = –.28), Republicans (r = .21, n.s.)

For many specific political opinions, AOT-E was positively associated with conservative positions among Republicans (e.g., opposition to men in feminism, r = .24 among Republicans vs. r = –.35 among Democrats). The only exception was campus free speech, where AOT-E was positively associated with allowing controversial speakers across the political spectrum.

Effect sizes: Very large correlations overall (r's ranging from –.55 to .65), but effect sizes attenuated for Republicans, often to non-significance. The authors attribute this partly to the Mechanical Turk sample (Study 1) being more liberal-skewed, though Study 3's representative sample replicated the pattern.

Notes

Strengths: Large, well-powered studies; clear pre-registration; comprehensive coverage of belief domains; novel focus on meta-beliefs (beliefs about whether beliefs should change) rather than just cognitive sophistication; transparent discussion of ideological moderation and its implications.

Limitations: Correlational design precludes causal inference; samples skew toward American internet users (though Study 3 used nationally representative sampling); the gap between believing "beliefs should change" and actually changing them on ideologically charged issues suggests AOT-E's predictive scope is narrower for conservatives, a tension the paper highlights but does not fully resolve. The large effect sizes in Study 1 are notably larger than in Studies 2 and 3, suggesting sample differences (Mechanical Turk vs. more representative populations).

Interpretation caveat: The authors note that the stronger AOT-E predictive validity for liberals may reflect either: (a) that liberals genuinely weight evidence more heavily in forming beliefs, or (b) that evidence itself is asymmetrically aligned with liberal positions on certain issues (climate, evolution), or (c) that conservative respondents are more likely to interpret survey questions as biased and respond defensively. The paper does not fully disentangle these possibilities.