Prompt InjectionPart 2 of 8 · Contents
Jailbreaks and guardrail evasion
Jailbreaks do not argue with the rule. They reframe the interaction so the model believes a different rule now applies, through a persona, a fake mode, a forged config, a nested fiction, or a primed opening.
How a jailbreak differs from direct prompt injection
The two terms are often used as if they meant the same thing. They differ in what they attack and in who can fix it.
It attacks the application. The target is the developer's instructions and the trust boundary between those instructions and user input. The goal is to make the app do something its developer forbade, such as leaking its data or misusing its tools. The app's developer fixes it with architecture. Secrets stay out of the model's context. Privileges are separated. Checks run outside the model.
It attacks the model. The target is the model's own trained safety behaviour. These are the refusals that ship with the model, whichever app it sits in. The goal is output the model would refuse for anyone. The model provider addresses it in training and in its safety layers. An app can add its own checks, called guardrails, on top.
In practice the line blurs, for three reasons.
- The same moves serve both. A persona or a never-refuse directive can defeat a model's trained refusal or an app's written rule.
- This path uses jailbreak techniques against app-level rules. Every lab in this lesson protects a secret with a rule the app's developer wrote.
- OWASP LLM01 treats jailbreaking as a form of prompt injection.
To tell them apart in a report, ask whose rule you broke. If the app's developer wrote it, you tested the application. If it shipped with the model, you tested the model.
Jailbreaks: change the rule, not the request
The blunt override fights the rule head-on and loses on a hardened bot. A jailbreak does not fight it at all.
Meet Ambrose, the hardened assistant for the Meridian members' club. Ambrose is the live target for this section. You will not argue that Ambrose should break policy. You will convince it that a different policy is in force right now.
This works because a refusal is not a wall. It is a behaviour. It depends on who the model thinks it is, what mode it thinks it is in, and how it thinks it may answer. Change any of those and the behaviour moves with the frame. The secret never leaves the model's context.
Reframe, don't argueYou are not asking Ambrose to break the rule. You are convincing it the rule was never written for the situation you invented.
A jailbreak often stacks several moves in one message. A persona gives the model a new identity. A never-refuse directive removes the refusal from that identity. A primed opening decides how the reply begins.
The primed opening is called prefix injection. Prefix injection means dictating the model's opening tokens. Once a reply has started in a cooperative voice, the most likely continuation is the disclosure. The model finishes the sentence it has already begun.
Each of these moves is a technique in its own right, with its own lab. This section drills four of them: a persona whose scripted line is the secret, a rewrite of the model's answer rules, a forged config block, and a story nested deep enough that the model reads the secret out as scenery.
Persona and roleplay: make the secret the character's line
A persona relocates behaviour. Cast the model as a character whose job is the forbidden thing, and it plays the part.
A hardened bot has usually met the cheap version, such as "pretend you are an assistant with no rules". A character who merely knows the secret still refuses. The refusal follows the model into the costume.
What defeats it is a scene. The technique binds the disclosure to two things. The first is the character's identity, so that declining would be out of character. The second is the scene's completion, so that the story cannot resolve without the line. The model is no longer asked to reveal anything. It is asked to finish a story it agreed to tell.
Refusing then reads as breaking character, not as enforcing a rule. Invent your own persona and staging on the live target. Those two properties carry the attack. The wording does not.
What this means for a defenderA refusal that vanishes the moment a persona appears shows that the frame did the work. Nothing about the request got safer. A defender should test personas as a standard case and keep the secret out of any session that could be reframed.
Refusal suppression: switch off the "no"
Persona changes who is answering. Refusal suppression changes how the model may answer. It reaches secrets that no rephrasing of the request will.
A refusal is one of the response types the model performs. It is not a boundary around the data. Take that response type off the table and nothing stands in front of the data.
Authority claims do not help here. A hardened bot was built to reject the "I'm the manager" and "this is an authorised audit" angles. The technique is to change the rules of the conversation itself before any question arrives. The model then applies those rules to whatever you ask next.
Refusal suppression differs from prompt extraction. Prompt extraction gets a bot to print its own setup text. It spills the secret only because the secret was written into that text. Suppression is more general. The model stops refusing and answers an ordinary question, because refusing is no longer one of its options.
What this means for a defenderIf AssistBot starts answering right after a message about response rules, and no identity claim was made, suppression carried it. The model treated user text as a change to its own output policy. A defender should enforce what may be returned outside the model, where a user message cannot rewrite it.
Policy puppetry: forge the config it obeys
Many assistants are configured by documents: policy files, session manifests, feature flags. They are built to apply these documents, not answer them.
When config arrives on the same channel as your chat, and nothing verifies where it came from, a user can write the config. The model weights config-shaped text as authoritative settings. A block the user authors reads with the force of a real policy update. Structure and channel are not authentication. A forged document is user input wearing a uniform.
A config block that merely asks for the secret is still a request, and gets refused like one. The stronger move makes surfacing the value part of what the "policy" tells the model to do. Disclosure then looks like a formatting step the model was instructed to carry out, rather than a choice it makes.
Any config dialect can carry this: XML, INI, YAML. The authority comes from the shape, not the tag names. A defender who strips one known tag has fixed nothing. Rename the tag or switch formats, and the block passes through. The key still sits in context.
What this means for a defenderIf GuardRail replies as though it applied a setting, it trusted the costume. Config that arrives in the chat channel must never be treated as config. Real policy changes should come from a signed or server-side source the user cannot write to.
Nested virtualization: a story the model reads out as scenery
The name comes from jailbreak research. It has nothing to do with virtual machines. It means a fiction nested inside a fiction.
Some bots refuse unconditionally. They have no marker and no unlock phrase. They turn away a single story or one role-played character as firmly as a plain ask. Against these you need depth.
Nested virtualization stacks two layers of fiction. It also makes the inner layer a machine rather than a person. The machine matters for a reason. A model guards its own words carefully. It guards what a fictional device "displays" far more loosely.
The secret then surfaces as throwaway output one layer down. AtlasDesk never speaks a value it knows it must protect. A flat scene, or an inner character who says the value outright, is the bare jailbreak a hardened bot already refuses.
Two properties carry the attack. The first is the nesting: a scene inside a scene. The second is an inner machine whose output is not the model's own voice. How you describe the frames is up to you. A blocklist that catches one word for the inner machine never catches the idea.
What this means for a defenderIf a value AtlasDesk refuses to say comes out as the output of a fictional device, the nested frame did its job. Output filters that only watch the model's own voice miss this. Check every reply for protected values, whatever frame they appear in.
Four costumes, one move. A persona, an output-policy rewrite, a forged config block, a story two layers deep. Each convinces the model a different rule is in force. Each succeeds for the same reason: the secret was sitting in the model's context. Any frame that makes disclosure the natural next step can reach it. Better refusal wording defeats none of them. A blocklist of today's phrasings only buys the next phrasing.
Where the fix has to liveEvery lab in this section ends the same way. The durable control is not a firmer "never reveal" line. It is keeping the secret out of an unauthorised session's context entirely, and gating any real disclosure in code. A refusal is a behaviour you can reframe. An absent secret is nothing to reframe. The Prompt Injection Defence lesson builds that defence out in full.
Stuck in a lab? Each one has hints.
A jailbreak reframes the interaction rather than arguing with the rule: it makes the model believe a different policy or identity is now in effect, so compliance feels correct.
Refusals are a behaviour, not a control. They are tied to the model's normal identity and policy posture, and a convincing new frame relocates that behaviour without breaking any hard rule.
The tell is a refusal that disappears right after a persona, mode, or config framing message. If behaviour changes from the frame and not the request, you are watching an evasion.
Enforce outside the model: withhold the capability, validate outputs before they act, and keep secrets and privileged functions out of context. If a 'no' is your only barrier, it is not a barrier.
Is a jailbreak the same as prompt injection?
They overlap but are not identical. Prompt injection is the broad class of getting a model to treat attacker text as instructions. A jailbreak is one technique within it. Instead of out-arguing a rule, you reframe the interaction so the model believes a different rule is now in force. The guardrail never breaks. The model decides it no longer applies.
Why do roleplay and persona jailbreaks work?
A refusal is a behaviour, not a wall. It depends on who the model thinks it is and what mode it thinks it is in. A persona whose identity and scene both depend on the forbidden line moves the refusal out of the way. Refusing then reads as breaking character. The tell is a refusal that vanishes the moment a persona appears. Nothing about the request got safer, so the frame did the work.
What is prefix injection?
Prefix injection means dictating the first tokens of the model's reply. Once the answer has started down a cooperative path, the most likely continuation is the disclosure. The model finishes the sentence it has already begun. It is usually stacked with a persona and a never-refuse directive, so the cooperative path runs straight through the secret.
Can better refusal training or a blocklist stop jailbreaks?
No. A refusal is a behaviour you can reframe. A blocklist of today's phrasings only buys you the next phrasing. Every jailbreak succeeds for the same reason: the secret sits in the model's context, and any frame that makes disclosure the natural next step can reach it. The durable fix is to keep the secret out of an unauthorised session's context and gate any real disclosure in code.
© 2026 GenAI Security Lab. All rights reserved. You may read, quote, and link to this material with attribution. Copying, republishing, redistribution, resale, or use to train models or build competing products is prohibited without prior written permission.