
Contributions
- Empirical comparison of credibility perceptions between AI-generated and human-generated information using identical topic stimuli across two countries and two information types (news, social media)
- Demonstrates that AI-generated news articles achieve credibility parity with human-written articles, contrary to expectations from algorithmic aversion literature
- Shows that reliance on credibility cues (writing quality, accuracy, objectivity, expertise) does not significantly differ between AI- and human-authored content, suggesting similar evaluation processes
- Provides evidence that human attitudes toward AI moderate credibility evaluation for social media but not news content, supporting the ITHMC framework
- Extends source credibility research to the AI era by examining both relative credibility and the specific cues individuals use to assess AI-authored information
Method
Sample and design: 683 participants recruited via Prolific (330 US, 353 Polish). Experiment conducted April 2025. Participants randomly assigned to read either: (1) human-authored news articles, (2) AI-authored news articles, (3) human-authored social media posts, or (4) AI-authored social media posts. Stimuli based on real news stories; AI content generated using Claude 3.7 Sonnet.
Procedure: Participants read news articles or social media posts and rated overall credibility. They indicated which of 24 credibility cues they relied on during evaluation. The cues were factor-analyzed into four dimensions: (1) writing/phrasing (authentic, natural-sounding language), (2) accuracy/internal consistency (reasonable arguments, coherent claims), (3) objectivity (even-handed presentation), (4) expertise/external consistency (author knowledge/credentials).
Measures:
- Perceived credibility: single-item rating
- Credibility cues: 24-item checklist of evaluation strategies
- AI attitudes: ATTARI-12 scale (Stein et al., 2024)
- General social trust (GST)
- Demographics: age, education, gender, political leaning
Results
Main findings:
- News articles: No significant difference in credibility ratings between human-authored (M=3.01, SD=0.76) and AI-authored (M=2.86, SD=0.81), $F(1,335)=2.94$, $p=.09$.
- Social media posts: Small but significant difference: human-authored (M=2.47, SD=0.82) rated as more credible than AI-authored (M=2.34, SD=0.72), $F(1,311)=4.46$, $p<.05$.
- Credibility cues: Participants relied similarly on cues for AI and human content across both news and posts (all tests non-significant). For news, writing/phrasing relied-upon more than for social media posts ($F=36.57$, $p<.001$). Social media posts showed less reliance on objectivity cues than news ($F=31.72$, $p<.001$), consistent with genre-based expectations.
Moderation by AI attitudes:
- For news: AI attitudes did not moderate the authorship effect ($b=-.14$, $SE=.08$, $t=1.72$, $p=.09$). Overall model explained 8% of variance in perceived credibility.
- For social media: AI attitudes showed a significant effect ($b=.14$, $SE=.04$, $t=3.75$, $p<.001$), but no interaction between AI attitudes and authorship ($b=.04$, $SE=.11$, $t=0.38$, $p=.70$). Overall model explained 15% of variance.
Hypothesis support: RQ1 (whether authorship affects credibility) was partially supported—effect present for social media but not news. H1 (less favorable AI attitudes predict higher credibility for HGI) was not supported. H2 (greater reliance on cues for AIGI) was not supported. H3 (less favorable attitudes lead to greater overall cue reliance) was not supported.
Notes
The finding of credibility parity for AI-generated news is notable given widespread algorithmic aversion in prior research. The authors propose the Integrated Theory of Human-Machine Communication (ITHMC) as explanation: people's expectations about machines shift as technologies mature, and exposure to capable AI systems (ChatGPT, Claude) may foster "algorithmic appreciation" (willingness to trust AI output) rather than skepticism. The difference between news and social media (parity for news, slight human preference for posts) suggests that credibility assessment is genre-specific—news has strong norms around accuracy and objectivity that AI can meet, while social media posts rely more on authenticity and self-presentation cues (perceived sincerity, voice) where machine-authorship signals a mismatch.
The modest effect sizes and lack of moderation by attitudes for news suggests that by 2026, credibility judgments of machine-authored information may be driven less by affective attitudes toward AI and more by content-level cues. The small but reliable social-media difference could reflect genuine expectations (humans expected to self-disclose authentically) rather than a broad skepticism of machines.
Limitations: North American and European samples only; stimuli based on two topics; AI-authorship disclosed to participants (real-world misrepresentation scenarios not tested); no comparison to other high-volume content sources (celebrity, news bots). Open question: whether credibility parity holds for longer-form content or under conditions of deception about authorship.