Contributions
- Tests multiple labeling strategies empirically: "AI-Generated," "Artificial," "Manipulated," "False," and "Misleading" versus control
- Distinguishes process-based cues (how content was made) from harm-based cues (whether content could mislead) in labeling design
- Measures separate outcomes: belief in claims, likelihood of engagement (sharing, liking, commenting, information-seeking), and image credibility
- Uses realistic social media post format with real AI-generated images that fact-checkers previously flagged as misleading
- Shows that different labels serve different informational functions; one-size-fits-all approaches may be ineffective
Method
Two pre-registered experiments with U.S. respondents recruited via Lucid Marketplace, quota-sampled to match national distributions on age, gender, race/ethnicity, and region.
Experiment 1 (October 2023, N=3,223):
- 5 conditions: unlabeled control + 4 warning labels
- 14 AI-generated images flagged as both misleading and artificially generated by journalists/fact-checkers
- Labels tested: "AI-Generated" (process-based), "Artificial" (process-based), "Manipulated" (process-based), "False" (veracity-based), "Misleading/[target audience]" (veracity/harm-based)
- Labels modeled on policy language from major platforms (Meta, Google, TikTok) and U.S. legislation (AI Labeling Act, Protecting Consumers from Deceptive AI Act)
Experiment 2 (December 2023, N=4,356):
- 3×2 factorial design: process cue (AI-Generated / Altered / none) × veracity cue (Misleading / none)
- 29 AI-generated images (including the 14 from Exp. 1), varying by topic (political vs. non-political), subject (religious leaders, celebrities, politicians), and societal importance
- Tests whether process and harm cues interact or operate independently
Measures:
- Engagement intentions: likelihood of sharing, liking/favoriting, commenting/replying, seeking information (0–1 Likert scale)
- Belief: respondents rated belief in core claims after a distractor task
- Image credibility: perceived believability, accuracy, authenticity, manipulatedness, and innovativeness (Exp. 2 only)
- Label evaluations: perceived helpfulness, informativeness, clarity
Analysis:
- Linear regression models with stimulus fixed effects and heteroskedasticity-robust standard errors
- Pre-registered hypotheses and pre-analysis plan; also conducted Bayesian mixed-effect models
Results
Effect on belief: All labels reduce respondents' belief in the synthetic content's core claims compared to unlabeled control (Exp. 1: −0.21 SD to −0.27 SD; Exp. 2: −0.21 SD to −0.34 SD). Harm-based labels and hybrid labels show stronger effects than process-only labels.
Effect on image credibility (Exp. 2): Labels reduce perceived credibility of the image (−0.33 SD to −0.39 SD). Process-based labels convey information about how content was made but are significantly less helpful for evaluating truth-value than veracity-centered labels.
Effect on engagement: Labels modestly reduce self-reported likelihood of sharing, liking, and seeking information about the post, but effects are much smaller than on belief (−0.07 SD to −0.17 SD across conditions). Engagement effects are particularly limited for process-based labels in isolation. Hybrid labels show larger engagement reductions (−0.28 SD to −0.34 SD) than either label type alone.
Subjective label evaluations: Respondents rated labels that explicitly identify content as AI-generated (e.g., "AI-Generated" and hybrid "Misleading/AI-Generated") as significantly more helpful for understanding how the content was made, compared to veracity-only labels (0.65 SD difference). However, the "Misleading" label alone received lower ratings for informativeness, suggesting it may feel vague without process information.
Interaction effects (Exp. 2): Both process and harm cues independently reduce belief and credibility. The provision of each cue further decreases respondents' belief in the post's claims and perceived credibility of the image; however, there is no multiplicative effect—the cues do not amplify one another beyond additive effects.
Notes
Strengths: Large-sample pre-registered experiments with diverse stimuli; clear manipulation of process vs. harm-based cues via factorial design in Exp. 2; realistic social media format and labels modeled on actual platform/policy language; careful measurement of multiple outcomes (belief, engagement, credibility) separating distinct mechanisms.
Limitations:
- Limited to survey context (single-post exposure); effects may be smaller in real social media feeds where warning labels are less salient or competing with algorithmic ranking
- Focused on AI-generated visual misinformation (images) flagged as misleading by fact-checkers; much AI-generated content circulates for entertainment without intent to deceive, and effects may differ
- U.S. English-speaking sample only; cross-national generalization uncertain
- Single post exposure prevents assessment of long-term habituation or dosage effects; prior work suggests labels may lose effectiveness with repeated exposure
- Scope limited to a curated set of label types; other approaches (e.g., more detailed explanations, source-credibility cues) not tested
- Detection and labeling accuracy assumed perfect; in practice, under-labeling or mis-labeling of content could negate or reverse benefits
Open questions:
- How do labels perform in field settings with competing visual/textual cues and algorithmic amplification?
- Do effects persist or habituate over multiple exposures?
- How do results generalize to non-English, non-U.S. populations?
- Which approaches—labeling, removal, demotion, or enforcement mechanisms—most cost-effectively mitigate AI-generated misinformation at scale?