Skip to content

Trusting expert- versus user-generated ratings online: The role of information volume, valence, and consumer characteristics

Computers in Human Behavior · 2013

In brief

An empirical study examining how Internet users evaluate the credibility and trustworthiness of online movie ratings depending on their source (expert critics vs. user-generated), information volume, and valence. The key finding: volume of user-generated ratings increases perceived credibility and reliance, whereas expert ratings remain credible regardless of volume; these effects are stronger among individuals who actively contribute to social media themselves.

Summary figure for Trusting expert- versus user-generated ratings online: The role of information volume, valence, and consumer characteristics

Contributions

  • Demonstrates that volume functions as a "warrant" or "signal" of credibility specifically for user-generated content, consistent with signaling theory and warranting principles
  • Shows that source (expert vs. user-generated) and information volume interact to shape credibility perceptions: users prefer experts at low-volume conditions, but favor user-generated information at high volumes
  • Extends theory by identifying receiver characteristics (individuals' own social media participation frequency) as a moderator of source-credibility effects
  • Empirical evidence that people's opinions and behavioral intentions conform to observed ratings, mediated by perceived credibility and volume

Method

Experimental design with 1,207 U.S.-based adults recruited via Knowledge Networks. Participants viewed a screenshot from a fictitious movie-rating website (boxofficepicks.com) showing ratings for a fictitious new film. The website displayed ratings manipulated along three dimensions:

  • Source: ratings attributed to lay moviegoers (user-generated) or expert movie critics
  • Volume: low (1 or 1–3 experts; 1 or 26 users), medium (3 or 26), or high (26 or 357 users)
  • Valence: low (2 out of 10), medium (5 out of 10), or high (9 out of 10)

This produced 18 experimental conditions. Participants rated the perceived credibility of the ratings, their reliance on them, confidence in their accuracy, congruence with their own opinion, and intention to see the movie. The study also measured participants' own frequency of contributing user-generated content online (blogs, forums, reviews, etc.) as a receiver characteristic.

Results

Hypothesis 1 (supported): For user-generated ratings, higher volume significantly increased perceived credibility ($F(2,562) = 19.21, p < .001$), confidence in accuracy ($F(2,562) = 8.62, p < .001$), reliance ($F(2,562) = 9.91, p < .001$), and opinion congruence ($F(2,562) = 3.84, p < .05$). No effect of volume on expert ratings. The relationship plateaued between medium and high volume (a "ceiling effect").

Research Question 2 (source × volume interaction, supported): A three-way interaction emerged: low-volume user ratings were perceived as less credible and less reliable than low-volume expert ratings ($F(2,1186) = 25.03, p < .05$); but at high volume, user-generated ratings produced greater opinion congruence than expert ratings ($F(1,1171) = 6.19, p < .01$). The main effect of source showed expert information rated as more credible and reliable overall.

Hypothesis 2 (supported): Positive valence (high ratings) significantly increased participants' personal movie rating ($F(2,1153) = 48.77, p < .001$) and behavioral intent to see the movie ($F(2,1153) = 3.72, p < .05$), consistent with informational social influence.

Hypothesis 3 (supported): Participants' own frequency of providing user-generated content moderated effects in Hypotheses 1–2 and RQ1. Those more active in social media provision showed:

  • Higher behavioral intent, credibility, reliance, and confidence across conditions (Set 1, $F = 179.24, p < .001$)
  • Smaller perceived difference between expert and user-generated information (Set 2)
  • Stronger alignment of their ratings with observed ratings (Set 3, $F(2,1153) = 98.32, p < .001$)

Notes

A landmark empirical contribution that isolates volume, source, and receiver characteristics as independent and interacting factors in online credibility judgment. The key insight—that aggregated user-generated information can equal or exceed expert information under conditions of sufficient volume—directly explains both why online reviews influence behavior and why crowds can generate credible consensus. The finding that active social media users are more susceptible to these effects is particularly relevant to understanding who may be most vulnerable to or trusting of user-generated misinformation at scale. The experimental manipulation of a realistic movie-rating site enhances ecological validity compared to laboratory designs.

Methodologically notable for a large, representative sample (1,207 participants) and careful operationalization of information volume (using realistic numbers: 1, 3, 26, 357 for users; 1, 3, 26 for experts, based on pretests). The absence of a head-to-head comparison (where participants saw both expert and user ratings simultaneously) is acknowledged as a limitation; the design instead tested them separately, reducing opportunity for direct comparative judgment.

How it connects

Similar papers

Discussed in GUIDE by

Summarized from the full text. Added 2026-11-20 from a submission by reza. Every statement on this page traces back to the source.