The task of identifying whether a given text passage was generated by an AI language model (such as GPT-2, GPT-3, or other models) rather than written by a human. This is a critical research area as language models become more capable of generating coherent, credible-sounding text.
Methods
Detection approaches include human evaluation, automated classifiers trained on statistical features (TF-IDF, linguistic patterns), zero-shot methods using model probabilities, and fine-tuned discriminative models (such as RoBERTa). Metadata-based approaches also exist, analyzing timestamps, digital signatures, and user behavior patterns associated with generation.
Key papers
- 2023-hanley-machine-made-media — Large-scale classifier study identifying synthetic articles across news websites using DeBERTa; documents sharp increase in synthetic article adoption by misinformation websites following ChatGPT release.
- 2023-mitchell-detectgpt — Zero-shot detection of LLM-generated text via probability curvature; achieves 0.95 AUROC without training data.
- 2019-solaiman-gpt2-release-strategies — comprehensive study of human and automated detection of GPT-2 outputs across model sizes