Nightshade: Poisoning Images to Fight Back Against AI
On this page
A poison that hides in a normal-looking image — and turns non-consensual scraping into a liability for the scraper.
| When | October 2023 |
|---|---|
| Target | Text-to-image models (e.g. Stable Diffusion SDXL) |
| Researcher | University of Chicago (SAND Lab) |
| Technique | Optimised clean-label, prompt-specific poison samples |
| Impact | <100 samples can destabilise a targeted prompt |
- 1An artist runs an image through Nightshade to add an invisible poison perturbation
- 2A model trainer scrapes the image (ignoring opt-out) and trains on it
- 3The model's association for the targeted prompt is corrupted
What happened
Nightshade, released by University of Chicago researchers in 2023, makes “poison” images that look visually identical to a normal image with a matching caption but corrupt a text-to-image model's learned association for a targeted prompt. Fewer than 100 poison samples can destabilise a single prompt in Stable Diffusion SDXL (e.g. poisoning “dog” so the model generates cats), and the effect bleeds to related concepts. It's framed as a way for artists to resist non-consensual scraping.
How it worked
The poison is an optimised, clean-label perturbation: the image and caption look correct, so it passes casual review, but it shifts what the model learns for that concept during training.
Root cause
Training pipelines ingest scraped data without integrity or poison detection, and even a tiny fraction of clean-label poison can move a targeted behaviour.
What this demonstrates
That model poisoning is cheap and effective at small scale — a handful of crafted samples in a huge dataset can reliably corrupt a concept.
How to defend a training pipeline
- Respect opt-outs and consent for training data in the first place.
- Filter and detect poison/anomalies before training.
- Track data provenance so a corrupted concept can be traced and removed.
FAQ
What does Nightshade do?
It produces optimised, clean-label poison images: visually normal with a matching caption, but they shift the model's text-to-image associations during training (e.g. poisoning “dog” so the model draws cats), with effects that bleed to related concepts.
How few samples are needed?
The researchers showed fewer than 100 poison samples can destabilise a single prompt in Stable Diffusion SDXL — and the effect can be stacked across prompts.
Is it an attack or a defence?
Both, depending on your seat: it's offered to artists as a deterrent against models trained on their work without consent, and it's a concrete demonstration that model poisoning is cheap and effective.
Sources
- Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models — arXiv / IEEE S&P 2024
- This new data poisoning tool lets artists fight back against generative AI — MIT Technology Review
- Nightshade, the tool that 'poisons' data, gives artists a fighting chance against AI — TechCrunch