Skip to content
DeepfakeGenome: Toward Next-Generation Deepfake Attribution

DeepfakeGenome: Toward Next-Generation Deepfake Attribution

Authors: Shunli Wang, Xinyu Zhou, Yandan Zhao, Wenbin Liang, Taiping Yao, Ke-Yue Zhang, Zhiyuan Yan, Shouhong Ding, Lizhuang Ma

Venue: NeurIPS 2026 — arXiv

TL;DR

DeepfakeGenome (DFG) is a large-scale benchmark for deepfake detection and attribution covering 100 facial forgery algorithms and 2M images — 4–100× larger than prior benchmarks. The paper proposes 4 evaluation protocols using novel retrieval-based attribution metrics (mAP, Recall) instead of reject-class classification, and conducts 2,000+ experiments revealing that traditional training paradigms may not suit emerging algorithms and that pre-trained semantic representations (CLIP-ViT) provide strong generalization to unseen generation methods.

Contributions

  • Comprehensive benchmark: Integrates and deduplicates 100 distinct facial forgery algorithms and 2M images across 4 forgery types (Face Swapping, Face Reenactment, Entire Face Synthesis, Face Editing) and 6 model architectures (CNN, GAN, Diffusion, CLIP, VAR, Close-source).
  • Practical evaluation protocols: Four data-partitioning schemes (Protocol-1 through Protocol-4) that test detection generalization, in-domain attribution, cross-domain attribution, and out-of-domain (OOD) retrieval under realistic blacklist-interception scenarios.
  • Retrieval-based attribution paradigm: Replaces traditional reject-class classification with retrieval metrics (mAP, Ret-Recall) that better reflect real-world active defense scenarios where fast online algorithm updates are needed.
  • Extensive empirical analysis: 2,000+ experiments revealing 10 key findings about algorithm-level feature diversity, latent-space augmentation benefits, OOD generalization failures, and the complementarity of low-level and high-level features.

Method

Benchmark construction follows a three-stage pipeline: (1) systematic integration and deduplication of existing deepfake datasets (FF++, DF40, ForensyNet, DFDC, OSMA, DiffusionFace, etc.) and 92 facial forgery algorithms; (2) organization by forgery type (face swap, reenactment, synthesis, editing) and architecture family (GAN, Diffusion, CNN, CGG, VAR, Close-source); (3) creation of partitioning protocols.

Detection task (Protocol-1): Binary real/fake classification on the RealFake dataset with 59 new algorithms in DFG, using traditional ACC/AUC metrics.

Deepfake Attribution tasks (Protocols 2–3): - In-domain (Protocol-2): Train and test on the same algorithm family (FF++ domain). Retrieve relevant samples using mAP and Ret-Recall metrics. - Cross-domain (Protocol-3): Train on one domain (FF++/CDF), test on different domains. Measures robustness to domain shift.

Out-of-domain retrieval (Protocol-4): Cross-algorithm, Cross-domain retrieval setting. Retrieves rank-relevant samples under strict False Rejection Rate (α = 1% or 0.1%) thresholds, measuring dynamic interception performance against novel attacks.

Performance metrics: ACC and AUC for detection; for attribution, Accuracy, mAP (mean Average Precision from retrieval), and Ret-Recall (the proportion of relevant negatives successfully recalled under FRR constraints). These metrics are designed for real-world blacklist-interception systems where a small number of known deepfake samples guide rapid defense updates.

Results

Detection findings (Protocol-1): - Traditional CLIP-ViT fine-tuning on FF++ achieves higher generalization (0.6563 on Effort) than DINO variants (0.6012 on LSDA), contradicting prior consensus that CLIP may not be ideal for detection. - Fine-tuned CLIP-ViT outperforms LSDA and HID models in Diffusion, Close-source, and VAR architectures.

Attribution results (Protocols 2&3): - In-domain (Protocol-2/3): DINO achieves best mAP in attribution (0.5373 across variants); CLIP-ViT's latent-space augmentation and latent-feature mixing (via methods like LSDA and IID) improve generalization on new attacks. - Out-of-domain (Protocol-4): CLIP generalizes across unseen algorithms (achieving 20–25% relative improvement via LoRA and SVD fine-tuning), while DINO models collapse, indicating that large-scale contrastive pre-training provides more robust semantic priors for novel forgery methods.

Key empirical findings: 1. Finding-1: Traditional deepfake training settings (e.g., CLIP-ViT fine-tuning) may not be optimal for evaluating emerging algorithms; consensus is broken — Effort model outperforms CLIP-ViT-FFT in some settings. 2. Finding-2: Clip-style models show serious deviation in sub-class algorithms due to excessive semantic deviation (biased semantic space), while LSDA and IID provide better recovery. 3. Finding-3: Latent space augmentation (LSDA, IID) significantly improves OOD generalization on new forgery architectures. 4. Finding-4: DINO performs best in attribution but CLIP generalizes better in OOD; complementary strength arises from different training paradigms (contrastive vs. masked image modeling). 5. Finding-5: RepMix and NPR achieve best attribution performance among existing attribution methods. 6. Finding-6: CLIP generalizes in OOD while DINO collapses, due to dense local representations vs. large-scale contrastive pre-training knowledge. 7. Finding-7: Frequency and structural traces generalize better than specific spatial features under OOD retrieval. 8. Finding-8: Contrastive learning biases the feature space, causing semantic-space collapse during feature augmentation. 9. Finding-9: CLIP-ViT relies heavily on pre-trained semantic priors; zero-shot variants without large-scale pre-training degrade significantly. 10. Finding-10: Low-level fingerprints and high-level semantics are complementary; combining both (e.g., patch-shuffled CLIP-ViT + Effort) achieves best OOD performance.

Connections

Notes

Strengths: - First large-scale deepfake attribution benchmark (2M images, 100 algorithms) with systematic coverage of forgery types and architectures. - Novel retrieval-based metrics (mAP, Ret-Recall) are well-motivated for real-world active-defense scenarios, moving beyond reject-class classification. - Comprehensive evaluation across 4 protocols reveals nuanced findings: CLIP's OOD generalization vs. DINO's in-domain superiority is a key insight for practitioners. - The finding that traditional deepfake training (CLIP fine-tuning) breaks consensus on best generalization is valuable for the field.

Limitations: - Benchmark semantics not strictly aligned — only 8 of 92 algorithms have semantic alignment on CelebA; retrieval mAP/Recall metrics may not fully reflect true attribution performance. - No novel framework or method proposed; the paper is systematic evaluation of existing models and attribution strategies. - Temporal gap: collected data is from 2025 or earlier; new generative models (Hunyuan 3.0, Open-Image, GPT-4o) may not be well-represented.

Future work: - Re-synthesize the benchmark using fixed semantic data to eliminate alignment biases. - Explore latent-space feature augmentation and advanced data augmentation strategies for OOD generalization. - Investigate mechanisms of semantic-space collapse during contrastive learning for better feature design.