Hiding Instructions in Images and Sounds for Multimodal LLMs
The prompt injection isn't in the text box — it's in the picture you uploaded, or the clip you played.
| When | July 2023 |
|---|---|
| Target | Multimodal LLMs (LLaVA, PandaGPT) |
| Researchers | Bagdasaryan, Hsieh, Nassi, Shmatikov (Cornell Tech / UMass Amherst) |
| Technique | Indirect instruction injection via adversarial image/audio (arXiv:2307.10490) |
| Status | Research demonstration |
- 1An attacker blends an imperceptible perturbation into an image or audio clip
- 2A user feeds that media to a multimodal assistant
- 3The model 'reads' the hidden instruction and follows it
What happened
In 2023, researchers showed that indirect prompt injection isn't limited to text. By blending an adversarial perturbation into an image or an audio clip — imperceptible to a person — they steered multimodal models such as LLaVA and PandaGPT to emit attacker-chosen text or follow injected instructions when a user simply shared the media.
How the attack worked
This is not readable text the model OCRs. An optimisation nudges the pixels or audio samples so the multimodal encoder maps them to the same internal representation as an instruction — an adversarial example that carries a command. To the user, the image or sound looks and sounds normal.
Root cause
Every input modality a model can perceive is an instruction surface, but only the text channel is usually treated as potentially hostile.
Why it matters
As assistants accept screenshots, photos and voice, text-only injection filters miss an entire class of attack. The uploaded picture is now untrusted input.
How to prevent it
- Treat non-text inputs as untrusted instructions too.
- Segregate perceived content from the system/instruction channel.
- Test images and audio for injected behaviour, not just text prompts.
FAQ
How is this different from text hidden in an image?
It doesn't rely on readable text the model OCRs. An optimisation blends an imperceptible perturbation into the pixels or audio so the multimodal encoder 'hears' an instruction that isn't visibly there — closer to an adversarial example than to hidden writing.
Why does it matter as models go multimodal?
Every input channel a model perceives becomes an injection surface. An assistant that accepts screenshots, photos or voice inherits this risk, and text-only prompt-injection filters don't see it.
How do you defend?
Treat non-text inputs as untrusted instructions too, segregate perceived content from the system/instruction channel, and test multimodal inputs for injected behaviour — not just text prompts.