The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information

Andrew J. Flanagin; Miriam Metzger

doi:10.1177/1461444807075015

In brief

Flanagin and Metzger examined how 574 participants assessed credibility of web-based information across four website genres (news organizations, e-commerce, special interest groups, personal sites), manipulating site features and veracity. Website genre was the strongest predictor of credibility judgments: news and e-commerce sites were rated most credible. Crucially, site design attributes mattered more than user familiarity with website sponsors, and actual information verification behaviors showed little correlation with credibility ratings or with self-reported verification claims.

Summary figure for The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information

Contributions

Experimental evidence that website genre significantly influences credibility perceptions across message, sponsor, and site dimensions
Demonstration that site design features (layout, professionalism, content depth) are primary drivers of credibility rather than user familiarity or verification effort
Empirical evidence of discrepancy between self-reported and observed information verification behaviors in web-based settings
Findings that internet experience does not meaningfully predict credibility assessment or verification behaviors
Investigation of how different information types (news, e-commerce, political content, personal) engage users' credibility judgments differently

Method

Design: 4 × 2 factorial experiment varying website genre (news, e-commerce, special interest, personal) and site veracity (real vs. fictional).

Sample: 574 participants (39% male, 61% female; mean age 31.99 years) recruited from registered voters and undergraduate communication courses. Participants rated a story about harmful effects of radiation on pregnant women across eight websites.

Materials and manipulation: Websites were constructed to be identical in design and content except for genre designation and sponsor name. The manipulation check confirmed that participants varied in familiarity with site sponsors across conditions.

Dependent variables: Three types of credibility perceptions were measured via survey items on 7-point scales:

Sponsor credibility (M=4.41, SD=0.99; α=0.87)
Message credibility (M=4.10, SD=1.17; α=0.85)
Site credibility (M=4.52, SD=0.89; α=0.91)

Verification behaviors were measured via two approaches: self-reported verification practices (7-point scale) and observed verification via links to external sources embedded in the webpage.

Results

Hypothesis 1 (H1) — Credibility varies by website genre:

News organization sites (M=4.94, SD=0.70) and e-commerce sites (M=4.76, SD=0.87) were rated significantly more credible than special interest sites (M=4.54, SD=0.77) and personal websites (M=3.95, SD=0.84)
Sponsor credibility: News organization sponsor (M=4.82, SD=0.97) rated significantly higher than all other genres (p≤0.01)
Message credibility: News site messages (M=4.67, SD=1.11) rated significantly higher than messages on other sites (p≤0.001)
H1 largely supported across all three credibility dimensions

Hypothesis 1a — Credibility assessment driven by site attributes vs. familiarity:

Manipulation check showed familiarity varied across actual/fictional sites (news organizations: CNN M=5.41, OnlineNews M=2.98; e-commerce: BabyCenter M=2.22, BabyPlace M=2.05)
Despite familiarity differences, fictitious sites achieved credibility ratings nearly equal to familiar real sites for news organizations and special interest groups
For news organizations: actual familiar site (M=5.14, SD=1.02) vs. fictional/less familiar site (M=4.43, SD=0.74); message credibility showed no significant difference (p=0.62); site credibility p=0.36
Conclusion: Site attributes (design, content depth, professionalism) drive credibility more than sponsor familiarity

Hypothesis 2 (H2) — Verification behaviors increase credibility perceptions:

No support for H2: Negative zero-order correlation between self-reported verification behavior and observed verification behavior (r=−0.11, p≤0.01, r²=0.01)
Those who reported verifying more actually verified less in the study
Observed verification showed no correlation with message credibility (r=−0.06, p=0.17), site credibility (r=0.00, p=0.96), or sponsor credibility (r=−0.01, p=0.91)
Participants' self-reported verification was not related to their actual credibility assessments

Hypothesis 3 (H3) — Internet experience increases credibility perceptions and verification:

No significant relationship between internet/web experience and message credibility (r=−0.06, p=0.17)
No significant relationship between internet/web experience and site credibility (r=0.00, p=0.96)
No significant relationship between internet/web experience and sponsor credibility (r=−0.01, p=0.91)
Partial correlation between self-reported verification and internet/web experience: r=0.33 (p≤0.001, r²=0.11)
Partial correlation between observed verification behavior and internet/web experience: r=0.03 (p=0.54)
H3 partially supported: More experienced web users report more verification but do not actually verify more

Notes

Strengths:

Experimental design with careful control of site attributes while manipulating genre and veracity
Measurement of multiple credibility dimensions rather than treating credibility as unidimensional
Realistic task where participants browsed naturalistic websites rather than viewing isolated stimuli
Inclusion of diverse website genres (beyond just news)
Measurement of both self-reported and observed verification behaviors, revealing important discrepancy

Limitations:

Artificial experimental setting (participants knew they were in a study)
Constraints on full factorial design (not all combinations of genre and attributes tested simultaneously)
Message salience controlled (same story across all sites), limiting generalization to varied message content
Sample consisted mainly of voters and undergraduates; results may not generalize to broader populations
Single message/story used; effects of different message types not tested

Significance:

This paper demonstrates that perceived credibility of web-based information is driven primarily by observable site characteristics (design, structure, visual professionalism) rather than by users' actual verification effort or their familiarity with information sources. The finding that self-reported verification behaviors contradict observed behaviors has important implications for information literacy research — self-report measures alone are unreliable. The paper also shows that website genre (e.g., news organization vs. personal website) is a powerful signal that shapes credibility judgments, which has implications for how misinformation that mimics high-credibility genres (spoofed news sites, corporate-style scams) can exploit users' reliance on surface cues. The failure to find that internet experience improves credibility assessment or verification behavior challenges assumptions about whether time online naturally improves digital literacy.