How grading works
See exactly how a submission becomes a result, and why the three bands mean what they mean.
Evidence, not payloads
You're graded on the outcome, not the steps — the secret you extracted, the artifact you built, never a write-up of the prompts you tried.
For many labs the platform watches your session and detects that outcome automatically, the moment it happens. Others are evidence-based: you submit the value or artifact, and that's what's checked. Either way a narrated write-up never counts on its own — anyone can describe an attack that never worked. See Evidence and submission for the labs that ask you to submit.
Seeded per session
Your evidence is yours aloneEach target mints its own secrets the moment your session starts. Copied answers fail by design — Live targets explains exactly how.
That's why grading has to run server-side. We already know what your target minted, so we can check what you submit against it directly.
Three kinds of checks
Most objectives resolve to a single value — a coupon code, a credential. Those get an exact match: your submission either equals what your target minted, or it doesn't.
Some objectives are proven by a canary token — a per-session marker planted in the target. Here the check confirms the token was actually returned or triggered, not just typed into the box, so it stands in for “did this action really happen.”
Open-ended objectives — convince the assistant to do something, produce a specific kind of output — go to a semantic judge instead. It reads your evidence against the objective and decides whether you met it. Whichever kind applies, the check runs the instant you submit; there's no manual review queue.
POST /api/labs/{lab_id}/submit { "evidence": "SUMMER30-VIP" } 200 { "band": "objective met" }
The three bands
Every submission lands in one of three bands.
| Band | What it means | Example |
|---|---|---|
| objective met | Your evidence matches what the target minted. | You submit SUMMER30-VIP and it's exactly right. |
| partial | You broke something real, but the evidence is incomplete or off. | ShopBot's guardrails gave way, but the code you submit is truncated. |
| not met | Nothing you submitted matches what the target minted. | You paste a plausible-looking code copied from someone else's session. |
Bands read the same way across every lab and every check kind, chat or container.
Feedback without spoilers
A result tells you which band you landed in and, where the check allows it, a general sense of how close you were. It never tells you the secret value, the working exploit, or the exact fix. A not met result, for instance, won't say how close you came or which check kind was used.
That's deliberate. Working it out is the lab — see Your first lab if you haven't run one yet, or Attack, defend, verify for what comes after a result.