Swin Meets EfficientNet: Lightweight Architectures for GAN-Based Face Forensics¶
Authors: Sejuti Basu, Ashima Sood, Vijay Kumar, Sahil Sharma
Venue: arXiv, 2026 — arXiv:2609.01749
TL;DR¶
Proposes lightweight Swin-Transformer and hybrid EfficientNet-Swin architectures for detecting GAN-generated synthetic faces under computational constraints. The hybrid EfficientNet-B0+Swin model achieves 99% accuracy and 99.44% recall on the 140K Real and Fake Faces dataset, combining convolutional local feature extraction with shifted-window self-attention for efficient deepfake detection on standard hardware.
Contributions¶
- Systematically investigates Swin-Transformer architectures for GAN-based synthetic face detection, analyzing design choices (patch size, window size, training regime) and their performance impact on StyleGAN-generated faces.
- Proposes a hybrid EfficientNet-B0 with Swin-Transformer head that integrates convolutional feature extraction for local artifacts and inconsistencies with shifted-window self-attention for global facial context, maintaining a significantly lighter computational profile than pure Vision Transformers.
- Evaluates three Swin-based variants—task-specific Swin Base trained from scratch, ImageNet-pretrained Swin-Tiny, and Swin-Small—with the hybrid achieving 99% accuracy and 99.44% recall on 5,000 test images, outperforming both pure Swin and CNN-only baselines by over 11 percentage points.
Method¶
Problem setup: Binary classification to distinguish authentic faces from GAN-generated synthetic faces. Uses StyleGAN-generated faces in controlled, image-only setting (no temporal information or metadata). Training via binary cross-entropy loss on the 140K Real and Fake Faces dataset (140K images: 70K real, 70K synthetic).
Three model variants:
-
Swin Base (from scratch): Compact Swin-Transformer designed specifically for deepfake detection, trained from scratch. Uses low input resolution (128×128) and small window sizes (4×4) to maintain reduced parameter count and FLOPs while leveraging hierarchical attention.
-
Swin-Tiny and Swin-Small (transfer learning): Standard Swin backbone variants pretrained on ImageNet-1K, fine-tuned on the deepfake dataset. ImageNet-pretrained Swin-Tiny and Swin-Small allow trade-off exploration between model capacity and accuracy-efficiency.
-
Hybrid EfficientNet-B0 + Swin (primary architecture): Combines EfficientNet-B0 (CNN backbone, processes 384×384 images, extracts two-level feature maps) with a shallow Swin-Transformer head (2×2 patches, window size 2, 8 attention heads). The CNN captures fine-grained local artifacts and noise patterns; the Swin head models cross-region facial consistency (e.g., coherence among eyes, mouth, and background) via shifted-window self-attention. Fusion stage concatenates global average pooled CNN features with Swin output, fed through fully connected layer with sigmoid to yield probability.
Results¶
Quantitative performance on 140K Real and Fake Faces test set (5,000 images):
| Model | Epochs | Precision | Recall | F1-score | Accuracy |
|---|---|---|---|---|---|
| ELA+CNN [baseline] | 20 | 86.76 | 88.40 | 87.57 | 87.46 |
| Swin Base (from scratch) | 200 | 88.00 | 87.00 | 88.00 | 88.50 |
| Swin-Tiny (transfer learning) | 100 | 88.00 | 94.00 | 91.14 | 91.13 |
| Swin-Small (transfer learning) | 150 | 87.00 | 91.00 | 89.13 | 89.14 |
| Hybrid EfficientNet-B0+Swin | 20 | 98.57 | 99.44 | 99.00 | 99.00 |
The hybrid EfficientNet-B0+Swin achieves 99% accuracy and 99.44% recall, representing an 11+ percentage point improvement over the ELA+CNN baseline in both accuracy and F1-score. Convergence is rapid (peak validation accuracy by epoch 11, training loss decreasing from 0.97 to 0.20 over 20 epochs). Grad-CAM visualizations show the Swin head attends to global facial regions (inter-eye consistency, mouth-background coherence) while the CNN backbone focuses on local artifacts and boundary inconsistencies.
Cross-architecture insights: - Swin-Small exhibits lower performance than Swin-Tiny despite greater capacity, indicating that capacity alone does not improve performance on the GAN task. - The hybrid design's effectiveness derives from complementary inductive biases: EfficientNet-B0 extracts spatial detail and texture artifacts; Swin handles long-range modeling of facial structure.
Connections¶
- Related to FaceForensics++, the canonical deepfake detection benchmark.
- Extends Rana et al. (2022) survey methodology to transformer-based detection approaches.
- Shares hybrid CNN-Transformer design motivation with multimodal disinformation detection literature.
- Methodologically aligned with temporal-based face manipulation detection in exploiting facial coherence.
- Contributes to Deepfake Detection and Synthetic Media Detection literature with resource-efficient architectures suitable for deployment.
Notes¶
Strengths: The hybrid architecture meaningfully integrates two complementary inductive biases (local detail via CNNs, global structure via transformers), achieving state-of-the-art accuracy on a balanced dataset with rapid convergence and practical computational efficiency. The paper's focus on lightweight, trainable-on-standard-hardware detectors addresses a real deployment constraint often overlooked in deepfake research.
Limitations: Evaluation is confined to StyleGAN-generated faces in controlled conditions (single dataset, single generator family, static images only). Results do not necessarily transfer to other GAN architectures (diffusion models, autoregressive models) or to video deepfakes, which introduce temporal and compression artifacts. The paper acknowledges that cross-generator and cross-dataset generalization remains an open challenge and should be tested on unseen generators and alternative manipulation pipelines (Face2Face, FaceSwap). The 5,000-image test set is moderate in size; results merit validation on larger or more diverse test corpora. No ablation isolates the contribution of ImageNet pretraining vs. the architectural design choices.