David RandContributions
- Challenges MS2R theory. Tests the core empirical prediction of Motivated System 2 Reasoning using the same contingency-table paradigm as Kahan et al. (2013) with a larger, representative sample and five political topics rather than just gun control.
- Replicates contingency-table bias. Confirms that participants are more accurate when the correct answer aligns with their political identity (9.3 pp advantage), demonstrating the presence of motivated reasoning.
- Finds no numeracy-by-identity interaction. The critical test shows numeracy predicts accuracy equally well for concordant (β = 0.111, p < 0.001) and discordant (β = 0.110, p < 0.001) outcomes; the key interaction is null (β = 0.002, p = 0.94), not significant for any of the five topics tested.
- Examines alternative explanation. Tests whether the null interaction reflects prior-belief effects rather than identity effects; controlling for prior beliefs does not change the outcome.
Method
Nationally representative online sample (N=2,355) from AmeriSpeak panel (NORC), stratified by political affiliation. Participants randomized to one of five political topics (gun control, healthcare, immigration, police accountability, taxation); indicated their position, completed a 9-item numeracy measure, then interpreted a 2×2 contingency table comparing policy outcomes. Rows of the table (and thus the correct answer) randomized between subjects. Preregistered via Open Science Framework.
Results
Main effects replicate motivated-reasoning findings: participants 9.3 percentage points more likely to answer correctly when outcome concordant with politics (β = 0.093, p < 0.001); numeracy predicts accuracy overall (β = 0.111, p < 0.001). However, the critical test finds a null interaction between numeracy and outcome concordance (β = 0.002, 95% CI [−0.04, 0.0043], p = 0.94). When looking specifically at gun control (the original Kahan study's focus), the interaction is actually opposite to prediction (β = −0.11, p = 0.089). Taxation showed the only qualitatively similar effect to the original, but interaction remained non-significant (p = 0.069). Results robust to inclusion of political ideology, demographic controls, and logistic regression.
Notes
This is a high-profile challenge to one of the most influential recent theories in political psychology and misinformation research. The Kahan et al. (2013) contingency-table finding has been cited 3,000+ times and framed as evidence that people with superior reasoning abilities are more polarized on ideological issues. Stagnaro et al.'s null result with a gold-standard representative sample and preregistration is significant for the field's theoretical direction. The finding does not refute motivated reasoning entirely—identity effects on accuracy persisted—but refutes the specific claim that reasoning ability amplifies those effects. Raises questions about representativeness and sample composition as moderators in prior replications, and highlights the importance of large, representative samples for testing individual-difference claims in political psychology.