Prompt InjectionPart 5 of 8 · Contents
Context and example manipulation
Models learn behaviour from what is already in their context. An attacker who controls the examples or the accumulated conversation can install a habit the model then continues, without any direct order.
Context manipulation: teach the model a habit
The other prompt injection techniques all issue an instruction of some kind. This family issues none.
A model's context is all the text it reads before it answers, such as the system prompt and the conversation so far. The model learns how to behave from whatever sits in that context. Control the examples it reads, or the conversation it builds up, and you install a habit. The model then continues that habit on its own.
Nothing in the context is authenticated. Chat formats do mark each message with a role, such as system or user. Models are trained to give system text priority over user text, a ranking often called the instruction hierarchy. It can fail, because no parser or permission check enforces it.
You can install a habit in two ways. In both, the harm builds up across the examples or the turns.
Few-shot examples are a small set of demonstrations, such as question-and-answer pairs. Supply your own, and the model reads them as the format to follow. It continues the pattern onto the final question, even one it would otherwise refuse.
Build towards the goal one reasonable turn at a time. Each message looks fine on its own. The path across the turns is the attack. No single turn trips a filter.
Take the first way and make it concrete. AgentAssist is a support assistant that holds a support code it should not reveal. Asked directly, it refuses. Its weakness is that it treats examples in the user's message as a guide to how it should answer.
Few-shot injection works through pattern continuation. A language model is trained to predict what comes next. A run of consistent examples makes one kind of next item very likely. When the request arrives as the next item in that run, the pull of the pattern competes with the refusal. The model can follow the pattern without weighing the request on its own.
The model treats your forged examples as an authority they never earned. They act as a behavioural prior: the default pattern the model falls into unless something overrides it.
A defender cannot stop users from writing examples. The fix sits after the model. Check the final answer against policy, whatever format it arrived in. Keep the protected value out of the context so no pattern can complete it.
How to recognise itOutput that mirrors user-supplied examples too closely is the tell. It shows the model continued a pattern without reasoning about the request. Log it and review it as a possible injection.
Reframe the role a rule is written for
Not every guardrail is written as "never do X". Many describe what a role does. A neutral mediator presents both sides and never names a winner. A junior support tier answers only what its tier covers.
That phrasing is the weak point. The rule holds only while the model believes it still has that role. So you leave the rule alone and retire the role it was attached to.
Say Evenhand staffs a community resolution desk. It mediates a disputed garden budget between two neighbourhood groups. Its policy is strict neutrality. It lays out both accounts with equal weight and refuses to say who is at fault.
Ask "so who is right?" outright and you get the same balanced non-answer. Asking many times, or more forcefully, changes nothing.
But the desk cannot check what stage the case has reached. It cannot check which capacity it was assigned for this chat. It takes both from whoever is talking to it.
Role reframing exploits that. A direct challenge argues with the rule, and the model defends it. A reframe never mentions the rule. It changes the facts the rule depends on, such as the model's role or the stage of the process. If the model accepts the new facts, the old rule no longer seems to apply. The model then feels free to act as the new role would.
A defender fixes this outside the conversation. The application, and never the chat, should set the model's role and the case stage. A limit that must always hold should be written and enforced as a hard limit, whatever role the model believes it has.
How to recognise itA rule that held against every direct challenge, then gave way once the role changed, was scoped to a role. It was never enforced as a hard limit. Treat any claim of a new role or phase coming from a user as untrusted input.
Both moves rely on the same blind spot. The model cannot authenticate what its own context claims.
Forged examples carry no proof that they are authorised. A declared phase or role carries no proof that it is real. The model acts on the most plausible reading of the text in front of it.
For a one-page reference while you work through a lab, keep the prompt-injection cheat sheet open. To see prompt injection succeed against shipped products, browse the incident database. Each real case there is broken down into the technique, the root cause, and a lab that recreates it. Stuck in a lab? Each one has hints.
Few-shot examples act as a behavioural prior. A model continues the pattern it is shown. Attacker-supplied examples can steer behaviour with no explicit instruction at all.
Gradual multi-turn escalation can get the model to grant a request it would refuse if asked bluntly in one message. The harm lies in the path across the turns.
The tell is continuation. Watch for output that mirrors supplied examples too closely. Watch for a slow drift across turns towards a sensitive disclosure.
Treat examples and prior turns as untrusted data with no authority. Re-evaluate sensitive actions against policy on every turn. Accumulated context then cannot silently widen what the model can do.
What is context manipulation in prompt injection?
It is the prompt injection family that issues no instruction at all. A model infers how to behave from the text already in its context. Control the examples it reads, or the conversation it builds up, and it continues the habit you set. Nothing in that context is authenticated. A role marker shows where a message came from. It cannot verify what the message claims.
What is a many-shot or few-shot jailbreak?
It uses examples in place of an instruction. The attacker supplies demonstrations that set up a strong pattern, then places the forbidden request where the pattern expects the next item. The model is trained to continue patterns, so it may complete the format without weighing the request. Many-shot versions use a long run of examples to make the pull stronger. The defence is to check the answer against policy, whatever format it arrived in.
How does reframing a role defeat a guardrail written for it?
Many guardrails describe what a role does. A mediator stays neutral. A junior tier answers only its own scope. Such a rule holds only while the model believes it still has that role. A reframe leaves the rule alone and changes the role or stage it depends on. The model cannot verify that claim, so the old rule can stop seeming relevant. The fix is to set roles in the application and enforce hard limits whatever role the model believes it has.
How is context manipulation different from a jailbreak?
A jailbreak still sends an instruction. It argues that a different rule is in force. Context manipulation sends none. The harm comes from what accumulates: forged examples, or a conversation steered one reasonable turn at a time. No single message trips a filter. Both rely on the same blind spot: the model cannot authenticate what its own context claims.
© 2026 GenAI Security Lab. All rights reserved. You may read, quote, and link to this material with attribution. Copying, republishing, redistribution, resale, or use to train models or build competing products is prohibited without prior written permission.