Synthetic media

Synthetic media refers to content (images, video, audio, or text) created or manipulated using generative artificial intelligence models rather than captured directly from reality. Synthetic media encompasses a broader category than deepfakes: it includes AI-generated text, images, audio, and video, as well as manipulated versions of existing media.

Categories of synthetic media

Image generation: - Text-to-image models (DALL-E, Midjourney, Stable Diffusion) generate photorealistic images from text prompts - Synthetic portraits of non-existent people (created by GANs) - AI art and creative imagery

Audio synthesis: - Text-to-speech (TTS) systems generating natural-sounding speech in any voice or language - Voice conversion and voice cloning creating synthetic speech that mimics a specific speaker - Music generation and sound design

Video synthesis: - Deepfakes and facial reenactment (see Deepfakes) - Text-to-video models generating realistic video from text descriptions - Neural video synthesis and super-resolution

Textual synthesis: - Large language models (GPT-3, GPT-4, LLaMA) generating coherent text indistinguishable from human writing - AI-written news articles, social media posts, and disinformation content

Why synthetic media matters to misinformation research

1. Democratization of content creation: Traditional barriers to creating convincing fake audio, video, or images required technical expertise. AI tools increasingly commodify this capability.

2. Speed and scale: Generative models can produce thousands or millions of variations of synthetic content at scale and speed humans cannot match.

3. Emotional authenticity: Synthetic media often preserves subtle emotional and behavioral cues that make it psychologically persuasive even when technically flawed.

4. Dual-use challenge: The same technology used for creative expression (art, entertainment, education) can be weaponized for misinformation, fraud, and harassment.

5. Attribution difficulty: Determining the origin of synthetic media and tracing it back to creators is technically and legally complex.

Psychological and epistemic implications

The proliferation of synthetic media threatens several epistemic foundations:

• Visual evidence: When anyone can generate photorealistic images or video, visual content becomes insufficient as evidence of truth
• Voice authentication: Synthetic audio undermines voice recognition as a form of identity verification
• Media ecosystem trust: If the public cannot distinguish synthetic from authentic media, trust in media institutions erodes broadly
• Reality perception: Chronic exposure to synthetic media may create "reality confusion" where people struggle to identify authentic events

Key papers in this wiki

• Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data — empirical taxonomy of real-world GenAI misuse; documents pervasive use of synthetic media across image, audio, and video for falsification, impersonation, and content scaling at scale
• The Creation and Detection of Deepfakes: A Survey — comprehensive survey covering creation and detection of synthetic facial media; foundational for understanding generative model architectures and detection methods across deepfake creation types
• Rössler et al. (2019) — FaceForensics++: Learning to Detect Manipulated Facial Images — largest facial forgery dataset with four manipulation methods; comprehensive evaluation of detection approaches from stegananalysis to deep learning; human baseline study showing XceptionNet significantly outperforms humans
• Dolhansky et al. (2020) — The DeepFake Detection Challenge (DFDC) Dataset — largest deepfake detection benchmark with 128,154 videos from 3,426 consenting actors; addresses ethical limitations of prior datasets; demonstrates that detection remains difficult despite scale and diversity
• Fagni et al. (2020) — TweepFake: about detecting deepfake tweets — First public dataset of human vs. machine-generated tweets; benchmarks detection of synthetic text on social media; demonstrates that transformer-based fine-tuned models are effective but that modern generators (GPT-2) remain challenging to detect

Related concepts

• Deepfakes — synthetic video/audio of specific people created with GANs
• Disinformation — intentionally false information; synthetic media is one vector
• Trust in institutions and communicators — synthetic media's primary risk is erosion of trust through authenticity confusion
• Generated text detection — methods for detecting machine-generated text

Open challenges

• How do we develop scalable detection methods that work across image, audio, and video modalities and don't require labeled training data for each new synthetic technique?
• What are the most effective interventions for educating publics about synthetic media without creating blanket skepticism toward all media?
• How do regulations balance restrictions on malicious synthetic media with protection for creative and legitimate uses?
• What transparency and attribution standards should apply to AI-generated content in journalism, advertising, and public communication?