Leveraging Frequency Analysis for Deep Fake Image Recognition

Joel Frank; Thorsten Eisenhofer; Lea Schönherr; Asja Fischer; Dorothea Kolossa; Thorsten Holz

In brief

GAN-generated face images exhibit distinctive artifacts in the frequency domain (DCT spectrum) that do not appear in real images. The authors show that simple linear classifiers trained on frequency coefficients achieve perfect separation (100% accuracy on test sets) and are significantly more robust to image perturbations than spatial-domain classifiers, demonstrating that frequency-domain analysis provides a reliable foundation for deepfake detection.

Contributions

Comprehensive frequency-domain analysis of GAN-generated images from multiple architectures (StyleGAN, ProGAN, BigGAN, SN-DCGAN), revealing consistent artifacts across different generators and datasets.
Identification of upsampling operations as the structural source of frequency artifacts: different upsampling strategies (nearest-neighbor, bilinear, binomial) produce detectable patterns in the DCT spectrum.
Demonstration that linear classifiers (ridge regression, kNN) trained on DCT coefficients achieve perfect or near-perfect accuracy while using significantly fewer parameters than CNN-based methods and exhibiting greater robustness to common image perturbations (blurring, compression, noise).

Method

The paper uses the 2D Discrete Cosine Transform (DCT) to convert images into the frequency domain. For each image, the DCT coefficients are computed and their statistics (mean, variance) are plotted as heatmaps. The key insight is that GAN-generated images show high-frequency artifacts—particularly a grid-like pattern visible as anomalous peaks in the upper-left and lower-right corners of the DCT heatmap—that are absent in real images.

The authors hypothesize these artifacts stem from the upsampling layers used in GAN architectures. In a low-dimensional latent space mapped to high-resolution output, upsampling operations (particularly those using strided transposed convolutions) create aliasing patterns. They verify this by training three GAN variants on the LSUN bedrooms dataset with different upsampling methods: nearest-neighbor, bilinear, and binomial (Zhang's anti-aliased kernel). More sophisticated upsampling reduces (but does not eliminate) the artifacts.

For detection, they train simple classifiers on log-scaled DCT coefficients: ridge regression, kNN, Eigenfaces-based methods, and a shallow CNN. On the FFHQ dataset, ridge regression achieves 100% accuracy; on CelebA and LSUN bedrooms, performance varies by generator but remains high. Importantly, they show that DCT-trained classifiers retain >85% accuracy even when test images are compressed, blurred, or corrupted with noise—far better robustness than spatial-domain classifiers.

Results

Detection accuracy (FFHQ dataset): 100% with ridge regression on DCT; 75.78% with ridge regression on raw pixels (24.22 percentage point gain).
Source identification: When multiple GAN models are candidates, the frequency-domain classifier correctly attributes images to the generative architecture with high accuracy across different architectures and datasets.
Robustness: Frequency-domain classifiers maintain ~85% accuracy under JPEG compression, Gaussian blur, and additive noise; spatial classifiers drop to ~60–65%.
Data efficiency: Training in the frequency domain requires significantly less data; models trained on only 20% of the training set retain ~85% accuracy on frequency coefficients, versus 98.57% for full data.

Notes

Strength: The frequency-domain insights are well-motivated and the root cause analysis (upsampling operations) is concrete and actionable. The robustness evaluation is thorough. Weakness: Scope is limited to face images; generalization to video or non-face content is not addressed. The paper does not evaluate adversarial perturbations designed specifically to evade frequency-domain detectors (addressed in later work, e.g., Carlini & Farid 2020). The assumption that GAN architecture persists at inference time may not hold for post-processed or heavily edited synthetic faces. Overall a strong methodological contribution that exposes a fundamental trade-off in how GANs map latent space to high-resolution output.