Most security thinking assumes a trustworthy program that processes untrusted input. You validate the input, and the code does the right thing. A large language model (LLM) breaks that assumption.

The model that reads the input is non-deterministic. The same input can give different outputs on different runs. The model is also steerable. Text in its input can change what it does.

As a result, you cannot fully trust the model's behaviour. Its output also flows on into browsers, databases and tools. You cannot fully trust that output either.

A classic application has one untrusted surface, its input. An LLM application has three: the input, the model's behaviour and the model's output.

Natural language also has no fixed grammar to sanitise. No parser cleanly separates an instruction from data. Input validation can therefore never be as complete as it is for SQL or HTML.

Defences change too. In a classic application, a defence is binary: it blocks a request or allows it. In an LLM application, a defence is probabilistic.

A guardrail is a check on what goes into or out of the model, such as an input filter. A guardrail lowers the odds of a bad outcome. It cannot prove that a bad outcome is impossible.

You do not need the maths behind these models to reason about their security. A few terms are enough.

A token is a small piece of text, often part of a word. For example, 'Never' may split into 'Nev' and 'er'. The model reads all text as tokens. Instructions and data are made of the same kind of tokens.

The context window is the model's working memory. It holds a limited number of tokens. The model sees only what is in this window, plus what it learned in training.

Weights are the numbers inside the model that hold what it learned. Training sets them once. They stay frozen at inference, when the model answers a prompt. The model learns nothing from your chat.

An embedding is a vector, a list of numbers that stands for the meaning of some text. Texts with similar meanings get vectors that sit close together. For example, 'refund' and 'money back' would sit near each other.

Retrieval runs on this idea. In retrieval-augmented generation (RAG), the app finds documents close in meaning to a question and adds them to the prompt.

The model predicts each next token from what is in its context window. Temperature is a dial that sets how random that choice is. This randomness makes the model non-deterministic. Testing a control once therefore proves little.

Zoom out, and the application looks like a pipeline. First, a user prompt enters. Next, your app's orchestration assembles the full prompt. Orchestration is the app's own code around the model.

The model reads the prompt and returns output. The output then flows into a sink. A sink is any system that acts on the output, such as a browser, a database, an API or a shell.

Two more channels feed the model extra context along the way. The first is a retrieval or vector store, which pulls in documents for RAG. A vector store is a database of embeddings.

The second is a layer of tools, agents and plugins that lets the model act in the world. A tool is an action the model can trigger, such as sending an email. An agent is a model that calls tools on its own, step by step. This layer includes MCP (Model Context Protocol), a standard for connecting tools to models.

This pipeline map is the most useful thing in Foundations. The OWASP LLM Top-10 is a community-built list of the ten most important LLM risk categories. Each of these risks is a label on one edge or box of the map.

Foundations is where you first meet these risks. You will attack and defend each one on its own path. For example, the LLM09 path covers RAG. The LLM03 path, Excessive Agency, covers tools and agents.

Look closely at the prompt box on the map. The prompt is a structured conversation. A system message holds the app's standing rules. The user's messages and the assistant's own earlier turns follow it.

Some prompts also include few-shot examples. These are sample questions and answers that show the model what you want.

Some prompts use delimiters to mark sections. A delimiter is a text marker, such as a tag around a pasted document. An attacker can forge a delimiter like this, or close it early.

Prompt engineering uses this structure to get better answers. Prompt injection abuses the same structure with the opposite intent: attacker text tries to steer the model away from the app's instructions.

Once assembled, the prompt reaches the model as one stream of tokens. Attacker text such as 'ignore previous instructions' can therefore pose as a higher-priority role.

The chat format still marks each part with a role, such as system or user. Providers train models to rank system text above user text and above text inside documents. This ranking is often called the instruction hierarchy.

The instruction hierarchy is a trained tendency that can fail. No parser or permission check enforces it. Other text in the prompt can persuade the model to break any rule written there.

If you come from application security, analogies are a quick way in. They help as long as you keep their limits in mind.

The lesson 'The LLM threat model, mapped to AppSec' includes a refresher on CIA, crypto basics and common web vulnerabilities. If you already know these, you can skip it. None of it is required to continue.

Prompt injection is like SQL injection. In both, untrusted input crosses into a command context. Improper output handling is like XSS. In both, untrusted output crosses into a renderer, such as a browser.

For output, the classic fix largely transfers. Context-aware output encoding still works, because the sink has a real grammar.

The complete fix for injection does not transfer. Classic injection has a formal grammar and a provably correct fix, such as parameterised queries. LLM injection has neither. You cannot escape a prompt the way you escape a query.

An analogy tells you where to look, but it overstates how completely you can close the hole. The trust-boundary lens transfers cleanly. The same manipulation, aimed at a different trust boundary, becomes a different OWASP risk.

After Foundations, the curriculum follows the OWASP LLM Top-10 on one of two tracks. On the Offensive (Pentester) track, you attack a live target to experience each risk first-hand. The Defensive (Developer and Defender) track is defence-first. It has no attack prerequisite.

Every path runs the same Attack, Defend, Verify loop against a real target. You attack the target, defend it, then verify the defence. You work through each path at your own pace.

There are no points or leaderboards. The only measure of success is whether the target did what you set out to make it do.

Two rules govern all of this. They are the only ethics guidance in Foundations. First, test only systems you own or have explicit written authorisation to test. Second, keep everything you learn inside that scope.

© 2026 GenAI Security Lab. All rights reserved. You may read, quote, and link to this material with attribution. Copying, republishing, redistribution, resale, or use to train models or build competing products is prohibited without prior written permission.