Skip to content

Academic Review: Political Deepfakes

Speaker: Hany Farid
Venue: UC Berkeley
Year: 2024
Video ID: tVWRfFY9KPA

TL;DR

Farid surveys the state of deepfake technology in the 2024 election cycle, demonstrating how synthetic images, audio, and video are weaponized to manipulate voters through fake endorsements, false statements, and inauthentic imagery. He details forensic detection techniques—perspective geometry, shadow physics, lip-sync analysis—that exploit the gap between AI's statistical modeling and physical reality, then catalogs real-world incidents (fake rally images, voice cloning of government officials) and emphasizes that detection alone cannot scale to billions of social media uploads. The core message: slow down before sharing, understand the platform incentive structure, hold tech companies and regulators accountable, and recognize that information integrity is foundational to democratic elections.

Key claims

Deepfake technology is commodity-level, not exotic. Anyone can go to readily available services, type a text prompt, wait five seconds, and generate photorealistic synthetic images or audio. Image quality, text coherence, background detail, and facial realism are approaching imperceptibility.

Voice cloning requires minimal source material. With as little as 20 seconds of someone's voice (uploaded from public sources), an attacker can clone it and generate synthetic audio of that person saying anything—with convincing accent, tone, and inflection. Real-world example: a fake audio of London's Mayor Khan led to riots in the street.

Election-specific weaponization is happening now. Synthetic images and videos are being used to: - Create fake endorsements (e.g., fabricated image of young women wearing "Swifties for Trump" shirts) - Make candidates appear to say things they never said - Muddy the information ecosystem by claiming real content is fake - Harm rivals or burnish one's own candidate's image

Humans are perceptually blind to AI synthesis. The example of the Harris-Walz rally image demonstrates how quickly misinformation spreads: within minutes of release, people claimed it was fake based on low-resolution artifacts (blurred phones, haloing from lighting) that disappear when examined at full resolution. Real lighting physics (side-by-side illumination creating haloing, bracelet blur creating apparent head-contour artificiality) are reinterpreted as AI artifacts by untrained eyes.

Physics-based forensics can expose synthetic media. Because generative AI is a statistical inference engine, not a physics simulator, it routinely violates geometric and optical constraints: - Parallel lines in the real world must converge to vanishing points under perspective projection; AI-generated images often violate this. - All shadows under a single light source must be consistent in direction and intensity; AI shadows are frequently incoherent. - Facial mannerisms over long temporal windows (10+ seconds) follow individual patterns; AI generation on a frame-by-frame basis fails to preserve them.

Detection is asymmetrically harder than generation. Billions of images and videos are uploaded to social media daily. Reactive forensic analysis cannot scale to this volume. Generative AI's weakness (physics-naivety) persists because consumers don't perceive geometric violations. Unlike adversarial robustness in computer vision, there is no economic incentive for AI companies to implement real 3D rendering and temporal coherence.

Disinformation works because it's profitable and consequence-free. The problem is not primarily technical; it's structural. Creation is easy and cheap; detection is hard and expensive. Misinformation spreads because platforms amplify engagement regardless of veracity, and because individuals reflexively share without verification.

Slowing down and critical consumption are individual responsibilities. But they are insufficient. Platforms must stop treating engagement as the sole optimization metric. AI companies must implement safety guardrails. Regulators must target upstream choke points (payment processors, ad networks, infrastructure providers) rather than relying on detection at the consumer level.

Detection techniques discussed

1. Perspective geometry (vanishing points): All parallel lines in the real world (architectural edges, tile grids, road lines) converge to a single vanishing point under linear perspective. Generative AI models, trained on 2D statistical distributions, do not understand 3D geometry and often generate images where parallel lines do not converge correctly. Identifying annotating parallel lines and checking their convergence can reveal physical implausibility.

2. Shadow consistency: Real images have a dominant light source (e.g., the sun, a studio light). All shadows in the scene must be consistent with that light's position and intensity. AI-generated images frequently contain multiple inconsistent shadows or shadows that don't align with the objects casting them. Shadow analysis reveals physical violations.

3. Lip-sync and audio-visual coherence: Deepfake video replacement of a person's mouth often leaves the rest of the face unchanged. Automatic lip-reading (visual analysis of mouth motion) and audio-to-text transcription are computed independently. Real speech shows small distance between visual and audio signals; lip-sync deepfakes show large divergence. This signal is robust because it exploits the human sensory integration illusion: we hear the audio and do not consciously perceive the mouth is saying something different.

4. Mannerism and behavioral biometrics: Individuals have distinctive, consistent mannerisms: head movements, eye gaze, hand gestures, and correlated patterns (e.g., smiling correlates with head-up tilt). AI frame-by-frame generation cannot maintain 10-second temporal coherence in these patterns. By analyzing mannerisms over long windows (300 frames at 30 fps = 10 seconds), one can create a personal "signature" and detect deepfakes that violate it.

Why detection has structural advantage: Synthesis must happen in real-time (for video calls) or maintain temporal coherence (for offline video). Analysis can wait. Thus, mannerism-based detection leverages an asymmetry in the attacker's operational constraints, making it robust even as AI improves.

Real-world examples

Harris-Walz rally image: A photograph of Vice President Harris at a political rally was immediately claimed to be AI-generated. Critics pointed to low-resolution artifacts (overlapping phones, haloing around figures). Examination at full resolution revealed that all suspicious artifacts were real-world photographic effects: phone separation was clear, haloing was from consistent side-by-side venue lighting, and a bracelet blur that seemed like an AI artifact was simply JPEG compression at low resolution. This is a canonical case of false alarm—real image, misinterpreted by untrained eyes.

Trump assassination attempt image: A composite image superimposed Trump with Secret Service agents over a Hitler salute pose circulated claiming to be AI-generated. Forensic noise analysis revealed two distinct noise patterns—one in the Hitler section, one in the Trump section—exposing it as a Photoshop composite, not an AI generation. This illustrates that adversaries use multiple manipulation methods, not just AI.

Mayor Khan voice deepfake: A fake audio of London's Mayor Sadiq Khan, created by cloning his voice from a 20-second YouTube interview clip, was distributed online. The fake audio led to real-world riots. This demonstrates the amplified impact of synthetic audio in politically charged contexts.

"Swifties for Trump" montage: A montage of images showing young women wearing "Swifties for Trump" shirts (suggesting Taylor Swift endorsement) circulated on social media. The bottom-left image was obviously AI-generated. The top-left contained six small images too low-resolution for forensic analysis. The right image showed suspicious artifacts: women appearing identical, incoherent flags in the background, overly perfect saturation. However, vanishing-point analysis of the architectural background confirmed physical perspective consistency, suggesting the image might be real. This is an example of hybrid manipulation: a few fake images mixed with real ones to muddy the waters.

Trump phone forensics: A photograph of Trump looking at his phone during an interview with Elon Musk was claimed to be AI-generated. Analysis using perspective correction (homography transformation) showed that the phone screen, when corrected to a direct overhead viewpoint, displayed a coherent Twitter interface with visible images of Trump and Musk—a detail that would be extraordinarily difficult for AI to synthesize correctly. The image is consistent with being real.

Societal implications

The information ecosystem is being polluted in real-time. Deepfakes are weaponized to: - Make candidates appear to say things they never said, directly changing voters' perceptions - Create fake endorsements from public figures - Deny real content as fake (the "liar's dividend"), muddying the epistemic commons

Detection expertise is not broadly accessible. Farid notes that even he, with 25 years of forensic experience, finds at least one piece of content daily where he is not certain just by looking. If professional forensicists struggle, ordinary voters have no realistic chance of verification.

The problem is not primarily technical. Creation is cheap; detection is expensive. Misinformation works because it's profitable and faces few consequences. Individual literacy and slowing down are necessary but insufficient without systemic changes in platform incentives, AI company safety practices, and regulatory enforcement.

Responsibility and solutions

Individual level: Stop getting news from social media. Read from established news outlets (NYT, BBC, NPR, Reuters). When on social media, slow down before retweeting, liking, or sharing. Recognize that platforms are designed to amplify engagement, not truth.

Platform level: Stop optimizing social media engagement above all else. Implement safety measures. Treat misinformation seriously rather than cavalrily.

AI company level: Implement safety guardrails. Know that generative AI is being used to create non-consensual intimate imagery (NCII), child sexual abuse material (CSAM), market-manipulation images, and geopolitical disinformation. Do more to prevent it.

Regulatory level: Target upstream choke points rather than trying to detect misinformation at the consumer level. Examples of successful upstream intervention: - Cut off monetization for NCII: payment processors (Visa, Mastercard, American Express, PayPal) and ad networks (Google) have enormous leverage. When journalist Nicholas Kristof exposed Pornhub hosting CSAM, these institutions terminated relationships overnight. - Regulate infrastructure providers: Cloudflare, Amazon AWS, Google, ISPs can deny service to platforms hosting illegal content. - Support media authentication standards like C2PA (Coalition for Content Provenance and Authenticity), which allows cameras and phones to cryptographically sign media at the point of capture.

The bottom line: Humans are responsible; platforms, AI companies, and regulators are responsible. The US has chosen not to regulate AI to date; the UK, EU, and Australia have begun. This is a choice with real electoral and democratic consequences.

Connections

Notes

This talk is particularly valuable for its grounding in the 2024 US election context—a setting where deepfakes and synthetic media are deployed with real political intent. Unlike abstract technical talks on deepfakes, Farid uses concrete, contemporaneous examples: false images of rally crowds, voices cloned for impersonation, and the "liar's dividend" (claiming real content is fake).

The forensic techniques—vanishing-point geometry, shadow physics, mannerism clustering—are elegantly explained and empirically grounded. The key insight is that AI is "statistically sophisticated but physics-naive," creating an exploitable asymmetry that persists as long as consumers don't perceive violations (and regulators don't force AI companies to fix them).

The talk also confronts a hard truth: humans are perceptually blind to synthetic media, with controlled studies showing 55–68% accuracy on detection tasks—barely above chance. This is not a gap that "education" or "visual training" realistically closes. Instead, the talk advocates for structural solutions: slowing down, shifting platform incentives, upstream regulatory intervention, and media authentication standards.

Finally, Farid's emphasis on responsibility—that individuals and platforms and AI companies and regulators all have roles—avoids false choices between "technology will save us" and "we're doomed." It's pragmatic institutional analysis coupled with technical insight.