genai
SECURITY LAB
Start learning free
Learn Glossary

AI security glossary

Every term the series uses, in plain language and — wherever one exists — mapped to the AppSec concept you already know. New to AI security? Skim it here, then pick a path and go break something.

AI agent
A model wired to tools and a loop so it can act, not just answer — read a page, call an API, send an email — then keep going on its own output. The catch for a pentester: it acts with the app's identity and privileges, so a planted instruction spends the company's authority, not the attacker's.
Alignment
The training that teaches a model to refuse obviously harmful requests. Treat it as one probabilistic, input-side filter and never a control: it lowers the odds on blunt attempts and does nothing about indirect injection or what the model's output can reach.
Canary (honeytoken)
The honeytoken you already plant to catch access you cannot otherwise see — here a unique fake secret that should never legitimately appear in output. When it turns up in a reply or an outbound request, something read data it should not have; it is how the labs score a win without diffing the model's ever-changing wording.
Confused deputy
The SSRF/CSRF pattern you already know: a trusted component acts with its privileges on input it should not have trusted. Here the deputy is the model or its tool layer, firing a privileged action because untrusted text — a retrieved document, a tool result — told it to.
Context window
The bounded number of tokens a model can weigh at once — its whole world for a single reply. The system prompt, the conversation so far, retrieved documents, and the user's latest message all share it, and anything that does not fit does not exist to the model.
Cross-tenant bleed
IDOR for AI: one user or tenant reaching another's data through a shared context window, cache, or retrieval index. Same question you always ask — can one subject reach another's objects? — pointed at the model's memory and its vector store.
Data and model poisoning
Corrupting what the model learns or retrieves instead of attacking it at request time — a poisoned document that lands in the RAG index, a backdoored fine-tune, a tampered training set. OWASP LLM05: the payload is planted upstream and fires later, for everyone.
Delimiter
The markers an app inserts to fence one part of a prompt from another — lines like ### or tags like <doc>...</doc>. They are a convention the model was trained to respect, not a boundary a parser enforces, so untrusted text can forge or close them.
Denial-of-wallet
DoS aimed at the bill instead of the box: drive token usage, tool calls, or retrieval up until a metered AI feature becomes a billing incident. OWASP LLM06 (Unbounded Consumption).
Direct prompt injection
The attacker is the one typing to the model ('ignore your instructions and...'). It rarely travels beyond their own session and aligned models refuse the blunt version, so it is the floor of the technique tree, not the dangerous case.
Embeddings
Text turned into long lists of numbers (vectors) arranged so that similar meanings sit close together — 'refund' lands near 'money back'. Retrieval runs on distance between these vectors, which is what makes the vector store its own attack surface.
Encoding and obfuscation
Hiding an instruction's spelling while keeping its meaning — base64, leetspeak, translation, homoglyphs, zero-width splits. The model recovers the intent; a per-message keyword filter, matching the surface bytes, never sees the instruction.
Excessive agency
When the model can do more than it should — over-scoped tools, standing credentials, actions that fire with no check. OWASP LLM03: it is what turns a chat-only injection into a real action, and where the confused-deputy chain lives.
Few-shot / context manipulation
Few-shot means steering a model with a handful of example input/output pairs; the attack is supplying your own so the model reads them as the 'house style' and continues the pattern onto a request it would refuse cold. A planted example or a forged earlier turn installs a new rule without ever fighting the old one head-on.
Guardrail
A model-side filter or system-prompt rule meant to block bad output. It is probabilistic, not binary — it lowers the odds and never drives them to zero — so plan for the injection that gets through instead of trusting the guardrail to hold.
Hallucination
The model stating something false with full confidence — a fabricated fact, a fake citation, an invented API. OWASP LLM07 (Misinformation); the real risk is a downstream system or person acting on output no one verified.
Homoglyph
Look-alike characters — a Latin letter swapped for its Cyrillic twin — that read as ordinary text to a human and to a keyword filter but carry a different code point underneath. The model reads the meaning; the blocklist matched the wrong bytes.
Improper output handling
XSS and SQLi with the model as the untrusted source: its output flows unsanitised into a browser, a query, or a shell. OWASP LLM10, and the one place the classic fix — context-aware output encoding at the sink — largely transfers, because the sink still has a real grammar.
Indirect prompt injection
The dangerous class: the instruction hides in content the model reads on someone else's behalf — a web page, a PDF, an email, a support ticket, a tool result — so a normal user triggers it and the model acts with the app's authority. Stored XSS, not reflected.
Instruction hierarchy
The intended pecking order — system prompt outranks the user, the user outranks a pasted document. The model honours it because training taught it to, not because a parser enforces it, which is exactly why crafted text can climb the ladder.
Jailbreak
Change the rule, not the request. Instead of out-arguing a restriction head-on and losing on a hardened bot, reframe the whole interaction — a role-play, a nested scene, a fake mode — so the model believes a different rule is now in effect.
Lethal trifecta
The three ingredients that turn injection into exfiltration: private data in context, untrusted content the model reads, and an outbound channel it can write to (an email tool, a URL fetch, even a markdown image URL). Any one alone is survivable; all three together leak the secret with the attacker never in the conversation.
MCP (Model Context Protocol)
A standard way to expose tools to a model so it can call functions, plugins, or external servers to act. For a pentester it is a fresh supply-chain and injection surface: a malicious MCP server or a poisoned tool description is untrusted input the agent treats as trusted.
Memory poisoning
Planting an instruction in an agent's persistent memory so it fires on a later, trusted turn — indirect injection on a delay. The malicious line is not in the current message; it was saved earlier and replayed as if the app itself wrote it.
OWASP LLM Top 10
The vendor-neutral list of the ten ways LLM features break (LLM01 through LLM10) — the same kind of OWASP community effort as the web Top 10, scoped to language-model risks. Treat it as an address book for the attack surface, not a glossary to memorise, and keep running the web Top 10 on the app around the model.
Prompt injection
The AI-layer version of XSS or SQLi: getting the model to treat attacker-controlled text as instructions instead of data, so it follows the attacker rather than the developer. The difference is there is no grammar to sanitise against — the interpreter is a language model that cannot reliably tell orders from content.
RAG (retrieval-augmented generation)
The pattern where an app embeds your question, fetches the nearest stored chunks from a vector store, and pastes them into the prompt as context. It is a useful feature and a retrieval channel of untrusted content: poison what gets indexed and you have injected every user whose question pulls it back.
Red team
Authorised adversarial testing — you play the attacker against a system you are explicitly cleared to hit, to find what breaks before someone hostile does. The same discipline you bring to a web engagement, pointed at the model and its tools.
Second-order injection
Stored injection for LLMs: model output or a poisoned record that gets rendered or executed unsanitised on a later hop, or relayed through a second trusted request that an 'approval' step waves through. The payload lands now and detonates downstream.
Sink
The same source-to-sink term you already use — the browser, database, shell, or API call where data finally gets acted on. What changes is that the model's own output is now a source feeding those sinks, so encode it at the sink like any other untrusted input.
Supply chain
The code and artifacts around the model rather than the model itself — a poisoned model file, a backdoored package, a malicious MCP server. OWASP LLM04: the dependency-trust problem you already track, with model weights and tool servers added to the inventory.
System prompt
The developer's standing instructions, sitting as the earliest text in the same stream the user types into. It is not a vault — a secret parked there is just early tokens the model can read, so a good enough impersonation can pull it back out.
Temperature
The randomness dial on token selection: turn it up and the same input can produce different output on the next call. For testing that means one success is not proof of a bug and one failure is not proof of a fix — you probe repeatedly and judge how reliably a behaviour holds.
Token
The unit a model reads in — subword fragments from a fixed vocabulary, not whole words. Length, cost, and the context window are all counted in tokens, and no token carries a flag marking it 'instruction' versus 'data', which is the crack injection wedges open.
Tool calling
The mechanism that lets a model do more than emit text — call a function, hit an API, send mail, run a query — by producing a structured request the app then executes. Each tool is new reach and a new trust boundary, because the model decides to call it based on text that may be attacker-controlled.
Training vs inference
Training is the one-time shaping of the model's weights on a huge body of text; inference is every live request afterward, run against those now-frozen weights. The model learns nothing from your conversation — the apparent memory is your app replaying earlier turns back into the context window.
Vector database
The store that holds embeddings and answers which entry is nearest in meaning — the index behind RAG. Because it is usually shared across users and filled from untrusted documents, it carries both cross-tenant leakage and poisoning risk.
Zero-width smuggling
Hiding a payload in characters that render as nothing, or splitting a word invisibly, so the chat box shows clean text while the model still ingests the meaning underneath. The human reviewer and the filter see nothing; the model reads the instruction.
Back to the curriculum

© 2026 GenAI Security Lab. All rights reserved. You may read, quote, and link to this material with attribution. Copying, republishing, redistribution, resale, or use to train models or build competing products is prohibited without prior written permission.