
Contributions
- Extends source credibility research by integrating source expertise (expert-generated, user-generated, hybrid) with message characteristics (sidedness), showing these factors interact rather than operating independently
- Demonstrates that two-sided messages boost credibility of less-trusted sources (hybrid) but have only marginal effects on highly-trusted (expert) or low-trust (user-generated) sources
- Introduces flexible thinking as a moderator of credibility evaluation, distinct from need for cognition; shows that high flexible thinkers value balanced information more, suggesting that cognitive openness is a key predictor of how people weight message characteristics
- Provides empirical foundation for understanding heuristic-based credibility assessment in diverse information environments where source signals vary widely (e.g., encyclopedias, reviews, collaborative platforms)
Method
The study used a 3 (information source: user-generated, hybrid, expert-generated) × 2 (message sidedness: one-sided vs. two-sided) factorial design with quasi-experimental manipulation. A representative sample of 463 U.S. Internet users (241 female, $M_{age} = 47.95$, $SD = 15.08$) from GfK's professional panel viewed encyclopedia entries presented as originating from one of three sources:
- User-generated: Wikipedia (entries contributed by any anonymous user without editorial review)
- Hybrid: Citizendium (entries authored by registered users, reviewed by experts prior to publication)
- Expert-generated: Encyclopedia Britannica (entries written by recognized experts worldwide, with editorial gatekeeping)
Entries were presented in one-sided (single perspective) or two-sided (balanced, both-sides) versions. The design included multiple encyclopedia topics (Shakespeare, Cobain, autism, Graves' Disease, net neutrality, flat Earth) to enhance generalizability. Dependent variable: perceived information credibility assessed with a single item ("How much do you believe this information?") on a 5-point Likert scale ($\bar{X} = 3.00$, $SD = 1.11$).
Moderators included:
- Flexible thinking (open-mindedness): 9-item Likert scale ($M = 3.66$, $SD = 0.48$, Cronbach's $\alpha = .75$)
- Need for cognition: 9-item scale ($M = 3.53$, $SD = 0.57$, Cronbach's $\alpha = .80$)
Results
Main effects (ANOVA):
- Information source: Highly significant, $F(2, 426) = 42.05$, $p < .001$, $\eta_p^2 = .17$. Expert-generated entries rated most credible ($\bar{X} = 3.70$, $SD = 1.01$), followed by hybrid ($\bar{X} = 3.02$, $SD = 0.97$), then user-generated ($\bar{X} = 2.55$, $SD = 1.03$). Post-hoc pairwise comparisons (Bonferroni) all significant ($p < .001$).
- Message sidedness: Significant, $F(1, 426) = 4.47$, $p = .035$, $\eta_p^2 = .01$. Two-sided messages ($\bar{X} = 3.14$, $SD = 1.18$) perceived as slightly more credible than one-sided ($\bar{X} = 2.88$, $SD = 1.03$).
- Interaction (source × sidedness): Significant, $F(2, 426) = 9.10$, $p < .001$, $\eta_p^2 = .04$. Two-sided advantage emerges primarily for hybrid sources: hybrid one-sided ($\bar{X} = 2.75$, $SD = 0.96$) vs. hybrid two-sided ($\bar{X} = 3.44$, $SD = 0.85$). No significant sidedness effect for user-generated or expert-generated sources alone.
Moderation analysis (hierarchical regression):
Block 1 (main effects): $R^2 = .032$. Block 2 (interactions with moderators): $R^2 = .079$. Final model: $F(7, 460) = 5.54$, $p < .001$, $R^2 = .079$.
- Flexible thinking moderation (H4): Significant interaction of sidedness × flexible thinking, $b = 0.152$, $t = 2.56$, $p = .006$, $\eta_p^2 = .14$. High flexible thinkers showed stronger preference for two-sided messages: slope for high FT ($+0.56$, $SE = 0.16$, $p = .001$) vs. low FT (not significant). Supports H4; flexible thinking is the only significant predictor of credibility at the interaction level.
- Need for cognition moderation (H3): No significant interaction, $b = 0.080$, $t = 1.37$, $p = .165$. Need for cognition did not moderate the sidedness effect, contradicting H3.
- Need for cognition as main predictor: Small positive effect, $\beta = .032$, $p = .365$, but non-significant when entered alone in Block 1.
Key finding: Flexible thinking is the only cognitive style to significantly moderate credibility evaluation under the conditions tested. High flexible thinkers interpret two-sided messages as more credible, particularly when source authority/reputation is ambiguous (hybrid sources). This suggests that openness to alternative viewpoints is a critical moderator of how message characteristics influence credibility judgments.
Notes
Strengths:
- Large, representative sample ($N = 463$) from professional panel representative of U.S. population; rare for experimental credibility research
- Rigorous factorial design with built-in replication across six encyclopedia topics (Shakespeare, Cobain, autism, Graves' Disease, net neutrality, flat Earth) enhancing ecological validity
- Clear methodological controls: validation across both real (Wikipedia, Britannica) and fictional (Peoplepedia, Userpedia, World Encyclopedia) versions; exclusion of participants who failed manipulation checks (42%)
- Novel contribution of testing flexible thinking as a cognitive style moderator, distinct from prior research focusing on need for cognition; finding suggests openness to multiple perspectives is more salient than general motivation to process information deeply
- Realistic context using online encyclopedias, a domain where source diversity is genuinely encountered
Limitations and gaps:
- Forced-exposure experimental design; credibility is measured in a single-item format after viewing. Real-world credibility evaluation involves active search, comparison, and temporal deliberation not captured here
- Single-item credibility measure (
Likertscale) may conflate believability with other dimensions (trustworthiness, authority, relevance); multi-dimensional approaches would clarify which aspects of credibility are affected by message sidedness and source - Topic effects noted but not deeply explored; Shakespeare entries rated differently from controversial topics (net neutrality), suggesting domain/controversy moderates results but this was not the focus
- Unclear whether results generalize to news, health information, or scientific claims where message sidedness has different epistemic weight (e.g., in science, balanced presentation of fringe views can harm credibility)
- Need for cognition operationalized and measured, but results show non-effect; future work should clarify why general motivation to think deeply does not predict credibility evaluation under these conditions (hypothesis H3 not supported)
Implications:
- For platform design: hybrid source models (community oversight, expert review, transparency) can increase credibility when paired with balanced information presentation; user-generated sources gain less credibility boost from sidedness alone, suggesting that institutional authority is a stronger cue
- For misinformation interventions: flexible thinking may be a cognitive target for prebunking or inoculation interventions, since people high in this trait are more receptive to alternative perspectives presented alongside mainstream information
- For information literacy: credibility training should emphasize metacognitive awareness of how source signals and message framing interact; the finding that high flexible thinkers benefited more from two-sided information suggests that encouraging openness to multiple viewpoints can improve evaluation under ambiguous source conditions