CNN-generated images are surprisingly easy to spot... for now

Sheng-Yu Wang; Oliver Wang; Richard Zhang; Andrew Owens; Alexei A. Efros

In brief

Despite the sophistication of GANs, CNN-generated images contain detectable fingerprints that persist across diverse generator architectures. Training a forensic classifier on images from a single GAN (ProGAN) generalizes surprisingly well to unseen architectures including StyleGAN, BigGAN, and DeepFake-generation models. The paper introduces ForenSynthis, a dataset of CNN-synthesized images from 11 architectures, and demonstrates that common image post-processing perturbations (blur, JPEG compression) improve robustness without harming generalization—though this advantage may not persist as generation methods evolve.

Contributions

Empirical demonstration that forensic classifiers trained on a single CNN generator exhibit surprising generalization to other architectures and datasets, contradicting the prevailing assumption that cross-generator detection is infeasible
Introduction of ForenSynthis, a dataset of synthetic images generated by 11 different CNN-based synthesis models spanning unconditional GANs, conditional GANs, and perceptual-loss models
Systematic analysis of factors enabling generalization: data augmentation (blur, JPEG), dataset diversity during training, and post-processing robustness
Evidence that common systematic artifacts in GAN-generated images (e.g., periodic patterns in frequency domain) provide exploitable forensic signatures

Method

The paper trains a ResNet-50 binary classifier (real vs. fake) on ProGAN-generated images, preprocessing both real and fake images with random horizontal flips, center crops, and optional augmentation (Gaussian blur at σ ∈ Uniform[0, 3], JPEG compression at quality ∈ Uniform[30, 100]). During training, the model is exposed to only ProGAN-synthesized images; at test time, it is evaluated on images from entirely different architectures (StyleGAN, BigGAN, CycleGAN, etc.).

The paper investigates cross-generator generalization through several ablations:

Effect of augmentation: models trained without augmentation show poor generalization; aggressive augmentation (blur + JPEG combined) provides the best out-of-distribution transfer
Effect of dataset diversity: training on subsets of ProGAN's images covering different classes (1, 2, 4, 8, 10, or 20 classes from LSUN) improves performance on other generators
Post-processing robustness: the model is evaluated under test-time perturbations (blur, JPEG, super-resolution) and shows varying degrees of degradation
Frequency-domain analysis: visualization of high-pass filtered images reveals that synthetic images share periodic artifacts (checkerboard patterns, grid artifacts) that differ systematically from real images

Results

When trained solely on ProGAN (100% AP), classifiers achieve surprising cross-architecture generalization:

StyleGAN: 99.6% AP
BigGAN: 87% AP
CycleGAN: 96% AP
DeepFake (face synthesis): 98.2% AP

Performance degrades gracefully under test-time perturbations (blur, JPEG): models trained with augmentation maintain 50–75% AP even at high levels of compression (e.g., JPEG quality 30), compared to 10–20% for unaggressive baselines.

Data diversity is beneficial: models trained on 20 LSUN classes generalize better (averaging ~90% AP across architectures) than those trained on a single class (averaging ~80% AP).

Comparison to prior work (Zhang et al.) shows that the paper's approach generalizes better to unseen architectures than specialized methods trained on specific generator pairs, with a mean AP improvement of ~8% across architectures.

Notes

The paper's central contribution—that CNN-generated images are surprisingly easy to detect when trained on a single architecture—is contingent on a static landscape of generation methods. The title's caveat ("for now") reflects the authors' awareness that as GANs improve their ability to evade frequency-domain artifacts (via architectural changes, training procedures, or post-processing), detection may become much harder. The observation that models generalizing well across ProGAN, StyleGAN, and BigGAN may not generalize to future architectures limits the practical lifetime of any deployed detector. The work is important for establishing that current CNNs do leave detectable fingerprints, but it implicitly motivates an adversarial arms race between generators and detectors.