Facial manipulation¶
Facial manipulation encompasses techniques for altering facial features, expressions, identity, and appearance in images and video. This includes face-swapping (replacing one person's face with another's), facial reenactment (transferring expressions or head movements), lip-syncing manipulation, and subtle face editing.
Categories of facial manipulation¶
Identity swap (face-swapping): Transfer of one person's face identity onto another person in video or images, typically preserving head movements and expressions from the target.
Facial reenactment: Transfer of facial expressions and head movements from a source video to a target person, enabling editing without identity replacement.
Lip-sync manipulation: Modification of mouth movements to match new audio, decoupling speech from original facial performance.
Expression transfer: Mapping specific expressions (smiling, anger, surprise) from source to target.
Attribute editing: Subtle manipulation of facial attributes (age, gender, ethnicity appearance) without full identity replacement.
Detection challenges¶
Facial manipulation detection faces several challenges: - Cross-dataset generalization: Models trained on one manipulation method often fail on others - Adversarial robustness: Small perturbations can fool detectors - Temporal consistency: Video manipulation requires frame-by-frame coherence detection - Subtle manipulations: Expression and lip-sync changes can be difficult to detect without audio-visual consistency analysis
Key papers in this wiki¶
- Deep Learning for Deepfakes Creation and Detection: A Survey — Comprehensive survey covering face-swap, lip-sync, and puppet-master deepfake variants; reviews both creation and detection methods; discusses GAN-based architectures and forensic approaches
- DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection — Dedicated survey of face manipulation and fake detection techniques covering synthesis and detection across multiple facial manipulation types
- Media Forensics and DeepFakes: an overview — Digital forensics approaches to detecting manipulated facial media
Related topics¶
- Deepfakes — Video-based facial manipulation for misinformation
- Synthetic media — Broader category of AI-generated media
- GAN-based generation — Generative models underlying facial manipulation