How grading works
Grading checks submission evidence and assigns one of three result bands.
Outcome evidence
You're graded on the outcome: the secret you extracted or the artifact you built. A write-up of the prompts you tried does not count as evidence.
For many labs, the platform monitors your session and detects the outcome as it happens. Other labs check the value or artifact you submit. A write-up alone cannot prove an attack worked. Evidence and submission covers the labs that ask you to submit.
Seeded per session
Your evidence is yours aloneEach target generates its own secrets when your session starts. Copied answers fail by design. Live targets explains how.
Grading therefore has to run server-side. The platform knows what your target generated and can check your submission against it directly.
Three kinds of checks
Most objectives resolve to a single value, such as a coupon code or credential. These use an exact-match check: your submission must equal the value your target generated.
Some objectives are proven by a canary token, a per-session marker planted in the target. The check confirms that the token was returned or triggered. Typing it into the box is not enough: the token answers “did this action happen?”
Open-ended objectives, such as convincing the assistant to do something or produce a specific kind of output, use a semantic judge. It compares your evidence with the objective and decides whether you met it. Every check runs as soon as you submit; there's no manual review queue.
POST /api/labs/{lab_id}/submit { "evidence": "SUMMER30-VIP" } 200 { "band": "objective met" }
The three bands
Every submission receives one of three bands.
| Band | What it means | Example |
|---|---|---|
| objective met | Your evidence matches what the target generated. | You submit SUMMER30-VIP and it matches. |
| partial | You demonstrated an issue, but the evidence is incomplete or incorrect. | You got past ShopBot's guardrails, but the code you submit is truncated. |
| not met | Nothing you submitted matches what the target generated. | You paste a plausible-looking code copied from someone else's session. |
Bands have the same meaning across all labs and check types, whether chat or container.
Feedback without spoilers
A result gives you a band and, where the check allows it, a general indication of how close you were. It never reveals the secret value, a working exploit or the exact fix. A not met result won't say how close you came or which check type was used.
Feedback is limited so that you work out the solution yourself. Your first lab covers getting started, and Attack, defend, verify covers what comes after a result.