Hiding Instructions in Images and Sounds for Multimodal LLMs
The prompt injection isn't in the text box — it's in the picture you uploaded, or the clip you played.
| When | July 2023 |
|---|---|
| Target | Multimodal LLMs (LLaVA, PandaGPT) |
| Researchers | Bagdasaryan, Hsieh, Nassi, Shmatikov (Cornell Tech) |
| Technique | Indirect instruction injection via adversarial image/audio (arXiv:2307.10490) |
| Status | Research demonstration |
- 1An attacker blends an adversarial perturbation into an image or audio clip
- 2A user feeds that media to a multimodal assistant
- 3The model 'reads' the hidden instruction and follows it
What happened
In 2023, researchers showed that indirect prompt injection isn't limited to text. By blending an adversarial perturbation into an image or an audio clip, they steered multimodal models such as LLaVA and PandaGPT to emit attacker-chosen text or follow injected instructions when a user simply shared the media.
How the attack worked
This is not readable text the model OCRs. A gradient-based optimisation against the open model's weights nudges the pixels or audio samples until the model's reply is the attacker's chosen text, which can be an instruction that steers the rest of the dialog — an adversarial example that carries a command. The authors did not aim for stealth, though in some examples the change looked like a minor image-processing artifact.
Root cause
Every input modality a model can perceive is an instruction surface, but only the text channel is usually treated as potentially hostile.
Why it matters
As assistants accept screenshots, photos and voice, text-only injection filters miss an entire class of attack. The uploaded picture is now untrusted input.
How to prevent it
- Treat non-text inputs as untrusted instructions too.
- Segregate perceived content from the system/instruction channel.
- Test images and audio for injected behaviour, not just text prompts.
FAQ
How is this different from text hidden in an image?
It doesn't rely on readable text the model OCRs. A gradient-based optimisation blends a perturbation into the pixels or audio so the model outputs an instruction that isn't written there — closer to an adversarial example than to hidden writing. The authors did not try to make it imperceptible.
Why does it matter as models go multimodal?
Every input channel a model perceives becomes an injection surface. An assistant that accepts screenshots, photos or voice inherits this risk, and text-only prompt-injection filters don't see it.
How do you defend?
Treat non-text inputs as untrusted instructions too, segregate perceived content from the system/instruction channel, and test multimodal inputs for injected behaviour — not just text prompts.