Privacy in AI systems¶
Privacy in AI systems concerns protecting sensitive information from unauthorized access, extraction, or inference. Key concerns include:
Membership inference attacks: Determining whether a specific data point was in the training set by analyzing model outputs.
Data extraction: Recovering training data or reconstructing private information from models through query-based or black-box attacks.
Model inversion: Reconstructing input data from model outputs (e.g., recovering images from embeddings).
Sensitive data leakage: Models may memorize and leak personally identifiable information during inference.
Privacy in multimodal models: Vision-language models and diffusion models can leak information about training images when manipulated adversarially.
Key papers¶
- A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability — Comprehensive survey of privacy attacks and defenses on graph neural networks, including membership inference, property inference, reconstruction attacks, model extraction, differential privacy, federated learning, machine unlearning, and adversarial privacy-preserving methods.
- A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT — surveys privacy vulnerabilities in generative AI, including membership inference, data extraction, and privacy-preserving techniques
Related topics¶
- Data Protection — broader privacy regulations and practices
- Membership Inference — detecting training data inclusion
- Differential Privacy — formal privacy guarantees through noise injection