Contributions
- Large-scale dataset of 15.46M articles (1.059M from misinformation/unreliable sites, 2.015M from mainstream reliable sites) classified as machine-generated or human-written
- DeBERTa-based detector trained on synthetic articles from multiple generative models (Grover, Article Forge, AI Writer, GPT-3.5, etc.) that generalizes across diverse news domains
- First large-scale empirical analysis of machine-generated article prevalence and differential adoption by website type
- Evidence that ChatGPT's release coincided with a marked increase in synthetic article usage on misinformation websites but not mainstream outlets
Method
The authors trained transformer-based classifiers (BERT, RoBERTa, DeBERTa) to detect machine-generated news articles:
Training data: Diverse dataset of articles generated from 10 different generative text models (Turing Benchmark, GPT-2, GPT-3, CTRL, XLNet, Transformer-XL, Article Forge, AI Writer, Grover, and GPT-3.5 via davinci API). Additional human-written articles from 2018 news sources. Total: 33,446 baseline human-written articles and up to 52,055 synthetic articles depending on model variant.
Detection models: Trained six DeBERTa configurations on different training datasets (Baseline, Pert., Para., Pert. + Para.) to account for adversarial perturbations and paraphrasing. Achieved average F₁ score of 0.959 and 0.992 precision, with conservative soft-max threshold of 0.50 for 0.7% FPR on the Signal Ads dataset.
Classification pipeline: Classified all articles published from January 1, 2022 to May 1, 2023 from 3,074 news websites, stratified by website popularity (Alexa rank). Filtered for English-language content and articles ≥1000 characters. Used Colly web scraper to fetch articles from website RSS feeds.
Results
Large-scale trends:
- Overall: 1.07% of 1,213,983 articles published in January 2022 were synthetic; by May 2023, fraction nearly doubled to 1.75% (825,541 synthetic articles), a 66% relative increase
- Mainstream websites: 57.3% relative increase in synthetic articles (0.51% → 0.87% absolute)
- Misinformation/unreliable websites: 474% relative increase (1.85% → 9.74% absolute)
ChatGPT impact: Noticeable jump in synthetic article prevalence on misinformation websites around late November 2022 (ChatGPT release). Analysis of ARIMA time-series interruption model confirms statistically significant immediate increase following ChatGPT's release.
Topic analysis: Misinformation websites show distinct topical preferences: Entertainment/World affairs/US/Nation topics in highest synthetic articles (Table 7). Mainstream websites show much higher odds ratios for Business topic synthetic articles; misinformation websites 2.96× more likely to generate Entertainment articles.
Website popularity: Surprisingly, smaller less-popular websites show larger relative increases in synthetic content (736% increase for Rank >10M) compared to large websites. Highly popular mainstream websites (Rank <10K) show only 0.70% absolute synthetic article prevalence.
Notes
Strengths: First large-scale systematic study of actual machine-generated article deployment across the news ecosystem. Training on diverse generator models improves generalization. Clear demonstration that misinformation sites systematically abuse generative models more than mainstream outlets. Detailed topic analysis reveals strategic use patterns. Robust detection model achieves high precision across multiple test datasets.
Limitations: Study restricted to English-language news. Absolute percentage estimates conservative (removed articles <1000 characters); actual synthetic content prevalence likely higher. Detection models trained on particular generator models; performance on future or unknown generators unknown. Cross-domain generalization tested but limited to particular news corpus. No verification of whether detected synthetic articles actually deceived readers.
Open questions: Whether detection methods remain effective against adversarial rewriting specifically designed to evade classifiers. How synthetic articles spread through social media vs. direct site visits. Whether fact-checking platforms and mainstream newsrooms are responding to synthetic content proliferation. Long-term trends as generator models improve and detection becomes harder.