Skip to content

Deep Learning for Deepfakes Creation and Detection: A Survey

Authors: Thanh Thi Nguyen, Quoc Viet Hung Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, Thien Huynh-The, Saeid Nahavandi, Thanh Tam Nguyen, Quoc-Viet Pham, Cuong M. Nguyen Venue: arXiv, 2019 — arXiv:1909.11573

TL;DR

Comprehensive survey of deep learning methods for creating and detecting deepfakes, covering face-swapping and lip-syncing generation techniques (GANs, autoencoders, encoder-decoder pairs), detection approaches (visual artifacts, temporal features, forensics), and challenges in cross-dataset generalization. The paper discusses implications for security, privacy, and misinformation propagation.

Contributions

  • Taxonomic review of deepfake creation methods, distinguishing face-swap, lip-sync, and puppet-master variants.
  • Systematic categorization of detection approaches: handcrafted features, deep learning classifiers, visual artifact analysis, and temporal features.
  • Survey of publicly available deepfake tools and datasets used for training and evaluation.
  • Discussion of forensic techniques (PRNU analysis, blockchain verification) for deepfake authentication.
  • Analysis of challenges: cross-forgery generalization, adversarial robustness, and the arms race between creation and detection methods.

Method

The survey reviews deepfake creation through multiple architectural approaches:

Creation methods: GANs (generative adversarial networks) with encoder-decoder pairs for face synthesis; StyleGAN and PGGAN for high-quality image generation; variational autoencoders (VAEs) for latent feature manipulation; and video-specific techniques using 3D convolutions and temporal consistency losses.

Detection methods: Classified into two major categories: 1. Handcrafted feature methods: Eye blinking frequency, facial landmarks, PRNU (photo response non-uniformity) sensor analysis, and optical flow-based inconsistencies. 2. Deep learning methods: CNN-based classifiers on handcrafted features, end-to-end deep networks (CNNs, RNNs, LSTMs for video), and artifact detection focusing on lighting inconsistencies, missing reflections, and geometric disparities.

The paper emphasizes the distinction between image and video detection, noting that video detection requires temporal consistency analysis across frames.

Results

The survey catalogs detection performance across multiple datasets (FaceForensics++, Celeb-DF, DFDC, UADFV) with varying success rates depending on forgery method and dataset. Key findings include: - Performance degrades significantly under cross-dataset evaluation (e.g., models trained on one forgery type fail on others). - Adversarial perturbations can fool deepfake detectors, highlighting the vulnerability of current methods. - Combining multiple detection approaches (ensemble methods) improves robustness. - Video-based detection outperforms image-only methods for lip-sync deepfakes.

Connections

Notes

Strengths: Comprehensive coverage of both creation and detection pipelines; clear taxonomy of deepfake types; extensive tool and dataset inventory; thoughtful analysis of cross-dataset generalization challenges. The paper situates deepfakes within information security and election integrity contexts, emphasizing the threat to democratic processes and the arms race between synthetic media creation and detection. Weaknesses: Some technical details are condensed given the breadth; detection results are from 2019 and predate more recent adversarial robustness work; limited discussion of audio-visual synchronization attacks and audio deepfakes. Remains foundational for understanding the state of deepfake technology at the time of writing.