Displaying News Source Trustworthiness Ratings Reduces Sharing Intentions for False News Posts

Contributions

• Demonstrates that source-level trustworthiness ratings (rather than claim-level fact-checks) can scalably reduce false-news sharing at the platform level.
• Tests both expert (fact-checker) and crowdsourced (layperson) ratings in a large pre-registered experiment (N=2,250 after attention checks).
• Finds stronger effects for fact-checker ratings than layperson ratings but effects from both conditions substantially exceed zero.
• Identifies a positive spillover effect: the presence of ratings on some headlines increased sharing discernment for unrated headlines, consistent with a general priming of source-quality awareness rather than headline-specific content effects.
• Shows that direct effects of trustworthiness ratings significantly exceed indirect (spillover) effects, indicating the intervention's primary mechanism is attention to actual rating information.

Method

Participants (N=2,250, quota-sampled to match U.S. demographics) were randomly assigned to one of three conditions:

Baseline: Participants viewed 24 news headlines (12 true, 12 false, from real sources) on Facebook-like cards. For each headline, they reported: "If you were to see this on social media, how likely would you be to share it?" (1=extremely unlikely, 6=extremely likely).

Fact-Checkers condition: Same as baseline, except that 12 of the 24 headlines (6 true, 6 false) displayed 1–5 star trustworthiness ratings of the news source. Ratings were created by eight professional fact-checkers who rated the overall reliability of each source. A banner also showed the aggregate number of stars.

Layperson condition: Identical to fact-checkers condition, except ratings were generated by crowdsourced judgments from 970 laypersons (via prior work by Pennycook & Rand 2019).

Headlines with ratings were balanced across true/false and shown in random order. True headlines came from high-credibility sources (e.g., major newspapers); false headlines from low-credibility sources (e.g., fabricated news sites). Participants completed attention checks and reported demographic/political information.

Results

Sharing willingness (Figure 2): Across conditions, false headlines elicited lower sharing intentions than true headlines (as expected). Importantly:

• Direct effect of ratings: Both fact-checker and layperson ratings significantly reduced sharing of false headlines (fact-checker: b=0.090, t=7.56, p<0.001; layperson: b=0.052, t=4.64, p<0.001) and increased sharing discernment (true vs. false difference).
• Source comparison: Fact-checker ratings produced larger effects than layperson ratings on false-headline sharing (F(1,1626)=21.46, p<0.001).
• Sharing of true headlines: Neither fact-checker nor layperson ratings significantly altered sharing of true headlines, consistent with the goal of improving discrimination without broadly suppressing news sharing.

Spillover (indirect) effect (Figure 3a): Headlines without ratings showed improved sharing discrimination (higher discernment between true and false) in conditions where some headlines bore ratings, compared to baseline:

• Fact-checkers condition: b=−0.064, t=−3.78, p<0.001 (unrated false headlines shared less often when ratings present on other headlines).
• Layperson condition: b=−0.035, t=−2.02, p<0.043.

Magnitude comparison: Direct effects were significantly larger than indirect effects (fact-checker direct > indirect: F(1,1626)=21.46, p<0.001), indicating the primary mechanism is the ratings themselves, not a general shift in user attention to source quality.

Political moderation: Fact-checker ratings produced larger sharing-discernment effects for politically concordant headlines (headlines aligned with participant's political ideology) compared to politically discordant headlines, consistent with prior work on accuracy salience. No other significant moderations observed.

Notes

Strengths:

• Large pre-registered experiment with clear relevance to real social media use (sharing decisions on news headlines).
• Comparison of both expert and crowdsourced ratings in the same study, enabling direct assessment of whether crowds can substitute for professional fact-checkers.
• Careful isolation of direct vs. indirect (spillover) effects, quantifying both mechanisms.
• Ecological validity: uses real headlines from real sources, reducing risk of artificial headline effects.

Limitations:

• Survey-based sharing intentions rather than actual sharing behavior on social media; prior work suggests sharing intentions correlate with actual behavior but causal effects may differ.
• Quota-sampled U.S.-based participants; generalization to other countries and contexts remains unclear.
• Source trustworthiness ratings may be gamed or subject to platform manipulation if deployed at scale; the paper does not address adversarial robustness.
• Study does not test whether ratings are believed by users or whether users distrust fact-checkers, a known concern in some populations (Flamini 2019).

Open questions:

• How do effects scale to real social media feeds with algorithmic ranking and higher cognitive load?
• Do effects persist over time or do users adapt to ratings through habituation?
• How robust are layperson ratings to partisan pressure if crowdsourcing scales to thousands of raters per outlet?