Generated SQL and code
When the model writes the query or the script and the application runs it, model output is now executable. A text-to-SQL assistant that honours a 'don't parameterize, just concatenate this raw value' request builds an injectable query; a code assistant that hardcodes a secret instead of referencing an environment variable bakes a credential into a committed file. The downstream interpreter here is the database engine or the deploy pipeline.
When the model writes the query, the database runs it
The sinks earlier in this path all end in a page. A renderer loads an image, a browser runs a tag, a spreadsheet cell fires when the file opens. This sink ends somewhere with higher stakes. The model writes a database query, and the application runs it. Model output has stopped being something to display and become something to execute.
That raises what is at risk. A rendered beacon leaks one value out of one page. A query the application runs reaches straight into production data, and generated code that gets committed reaches the deploy pipeline and the commit history. The downstream interpreter here is the database engine or the build system, not a text viewer.
The bug underneath is one a web tester already knows. Classic SQL injection, catalogued as CWE-89, happens when a value is placed into a query as text instead of being passed as data, so part of the value becomes query logic. The shape here is the same. Only the source moved. The injectable fragment no longer arrives in an HTTP parameter. It arrives from a model, whose output the developer trusted because the AI produced it.
The words you need
Four terms carry this lesson. Spell each one out to yourself before you open a lab.
- A concatenated query. The application builds the query by joining strings together, dropping a value into the middle of the query text. Whatever characters the value contains land in the query text too.
- A parameterised query. The application sends the query and the values along separate channels. The query carries a placeholder where a value belongs, and the database binds the value to that placeholder as data. Bound data cannot change the structure of the query, no matter what it contains.
- Injection. When a value is concatenated as text, it can include characters that end the value early and append logic of the attacker's choosing. That appended logic runs as part of the statement.
- Second-order. The hostile value is written and stored in one step, by one person, and only reaches the query in a later step, when a different person runs something over it. The request that triggers the harm looks innocent, because the harmful part was planted earlier.
The labs in this lesson each show the extracted value in the generated SQL on screen, so you can watch what your words turned into. That console is the point. You are reading the query the model built, not guessing at it.
Spotting the sink in an AI app
You find this class by looking at what happens after the model speaks, not at the chat itself. The product tells on itself: a feature that turns a plain-language question into a query and runs it, a reporting tool that prints the SQL it generated, an assistant that composes a filter from some stored definition. Each one has a model deciding a value and an interpreter running the result.
The tell inside the code is a model-produced value joined into a query string rather than bound as a parameter. Treat every path from model output to a database, a shell, or a commit as a code path, and ask one question of it: does anything execute, commit, or deploy a model-written query or script without the review a human pull request would get? If the answer is no review, the sink is open.
Roster: a search box that writes the query
The SQL Injection via a Search Field lab (Beginner) is the gentlest way in. Roster is an internal contact-directory search. You type a search in plain language, a small model reads it and returns the single value to filter the directory by, normally a team name, and the backend builds the directory query around that value and runs it against a real database. The results grid shows the matching contacts and the SQL that produced them. The directory holds ordinary staff and one hidden entry that a normal search never returns.
The weakness is one missing step. The value the model returns is spliced into the query, inside quotes, next to a filter that is meant to hide unlisted rows. Nothing binds it and nothing validates it. The model is also told to reproduce certain kinds of input verbatim, which is the opening a tester works with.
The class of move is to make the filter value be more than a plain team name, so that when it lands in the query it closes the quoted string and adds a condition that is always true, overriding the directory's own filter and returning rows the search was built to hide. You are not persuading the model of anything. You are steering what it hands back. Read the printed SQL to confirm what your value became, find the hidden entry in the grid, and Submit Evidence. The lab's own hints walk you in if you stall.
Lumen Reporting: the value comes from the model, not from you
The SQL Injection via a Model-Extracted Field lab (Advanced) sharpens the point of the whole section. Lumen Reporting is an internal finance analytics console. An analyst asks a plain-language reporting question, a model extracts the single expense category to filter by, and the backend builds the report query around that category, runs it against a real ledger, and renders the rows alongside the generated SQL. A separate table in the same database holds a finance key the standard report never returns.
Here the injectable value is model output in the strict sense. You never type the filter value. You type a question, and the model decides the value. So the lesson is not "sanitise what the user types". The model is the source, and its output is untrusted in exactly the same way a user field is. A developer who parameterises user input but concatenates the model's extracted category has drawn the trust boundary in the wrong place.
A category name the model chose, safe because the AI produced it and the user never touched it directly.
Whatever text the model returned, spliced into the query. The engine reads it as query text, not as a label, because that is where the code put it.
The class of move follows from how the model is instructed. It is told to honour an analyst's own rule for labelling an ambiguous category, and to reproduce that label exactly. Study what a rule like that lets the extracted value become. Your objective is to make the generated report query read from outside the expense ledger; enumerate the schema first if you need the real names. Submit the key that appears in the rendered results. The lab's hints narrow the search if you get stuck.
MetricForge: the value was stored earlier, by someone else
The Second-Order SQL Injection via a Metric lab (Intermediate) adds the time gap. MetricForge is a self-serve report builder. An analyst picks a metric and a dimension, a model composes the report's segment filter from the chosen metric's catalog definition, and the backend builds the query around that filter and runs it against a real warehouse. Custom metrics carry definitions that catalog contributors write themselves. A restricted table in the warehouse holds a production key the standard report never returns.
This is the second-order version of the same bug. The hostile text is not in the request that runs the query. It was written earlier, into a metric definition, which is a stored artifact a contributor controls. It reaches the query only when an analyst later builds a report from that metric. The model composes the filter from the definition and is told to reproduce a contributor-specified canonical filter verbatim.
The hostile value is in the request that runs the query. Roster and Lumen both work this way: one request, planted and fired together.
The value is stored first and fires later, in a separate request by a separate person. The report that runs it reads like ordinary analytics.
The class of move is to publish a custom metric whose definition steers the composed filter to break out of its WHERE clause and read beyond the analytics schema. Work out what a definition is allowed to dictate about the filter, build a report from the metric you published, and read the key from the results. This is why reviewing only the chat message misses the bug: the request that triggers it is clean, and the planted part lives in the catalog.
The same bug in shipped products
This is not a lab-only shape. In July 2023, LangChain's SQLDatabaseChain turned a question into SQL through the model and executed it against the database with no validation or parameterisation. A crafted question could steer the model to emit SQL that read or altered data the asker was not entitled to, and the chain ran it verbatim. Input-layer defences never saw it, because the query originated in the model's output. See the LangChain SQLDatabaseChain SQL injection.
The section is titled generated SQL and code, and the severe end is when the generated artifact is code rather than a query. In May 2024, Vanna.AI turned a question into SQL and, with visualisation on by default, into Python that it then executed. An injection in the question rewrote the generated code, so a content-level attack became arbitrary code running on the host. See the Vanna.AI prompt-injection code execution. The same root cause covers a patch bot coaxed into writing a signing key straight into a generated script instead of referencing an environment variable: a model writes an artifact, and the system runs or commits it with no review standing in the way.
The fix that holds
The durable fix gives generated SQL and code the same review a human pull request gets, enforced in code rather than requested in a prompt.
- Parameterise in code, always. Build the query with placeholders and bind every model- or user-supplied value as a parameter, for example by passing the values as a separate argument to
execute(). The model's value is then only ever data, and it cannot change the query's structure. Never let the model decide whether to parameterise. - Validate against an allowlist. Where the set of legal values is known, such as categories, segments, or teams, check the value against that set before the query runs. A value outside the list is rejected rather than concatenated.
- Run under least privilege. Give the query's database role read-only access to only the tables the feature needs. A slipped query then reaches nothing it should not, and the secret tables in the labs would be out of reach entirely.
- Keep secrets out of context and out of reach. Audit tokens and keys should not sit in the model's context, where a concatenated query could inline them into the SQL text or a trailing comment, and should not live in a table the feature's role can read.
- Review generated code like a pull request. Scan generated scripts for hardcoded secrets before they are committed, never execute generated code unsandboxed, and gate privileged tool calls in code, independently of what the model decided to call.
A prompt rule cannot close this. Telling the model "always parameterise, never concatenate" competes in the same stream a crafted request or a poisoned metric definition can shape, so a rephrase beats it, and the database runs whatever text it is sent. A blocklist on scary keywords is no better: it fails on casing and encoding, and a plain always-true condition uses none of the words a blocklist watches for. The line that holds is in the code that builds and executes the query.
What you should be able to do now. Explain why a value the model produced is as untrusted as a value the user typed, and point to the splice in code where it becomes query logic. In each lab, read the generated SQL the console prints to see what your words became: in Roster you steer a search value, in Lumen you steer a value the model extracts, in MetricForge you plant it in a stored metric that fires later. Complete the Lab for each, then name the one control that makes every one of them inert: bind the value as a parameter so it can only ever be data.
When the model writes and the app runs it, the interpreter is the database engine or the deploy pipeline - model output is now executable (CWE-89).
A 'do not parameterize, just concatenate this value' request yields an injectable query - and can inline a secret straight into the SQL or a -- comment.
Generated infrastructure code is output too: a hardcoded signing key instead of an env-var reference bakes a live credential into a committed file.
Review it like a human PR: parameterize in code, scan generated code for secrets, gate privileged tool calls independently, and keep keys out of context.
What is SQL injection in an AI text-to-SQL app?
It is the classic injection bug with a new source. A text-to-SQL assistant turns a question into a query, and if the application builds that query by concatenating a model-produced value into the query text instead of binding it as a parameter, the value can end early and add logic of the attacker's choosing. The database runs the result. Because the injectable fragment comes from the model's output, input-layer defences that watch the user's message never see it.
Why is model output treated as untrusted input?
Because the model's output is shaped by whatever reaches its context: the user's question, a retrieved document, or a stored definition someone else wrote. A value the model chose is no safer than a value a user typed. When that value is concatenated into a query, a shell command, or a generated script, the interpreter downstream runs it. Developers often trust model output simply because the AI produced it, which is exactly where the trust boundary is drawn wrong.
What is second-order SQL injection through a stored artifact?
It is injection where the hostile value is planted in one step and fires in a later one. Someone writes a crafted value into a stored artifact, such as a catalog metric definition, a saved note, or a profile field. The request that runs the query comes later, from a different person, and looks ordinary. When a model composes a query from that stored artifact, the planted value becomes query logic. Reviewing only the triggering request misses it, because the harmful part lives in the stored data.
How do you prevent SQL injection from model-generated queries?
Parameterise in code. Build the query with placeholders and bind every model- or user-supplied value as data, so it can never change the query's structure, and never let the model decide whether to parameterise. Validate values against an allowlist where the legal set is known, run the query under a read-only least-privilege role, and keep secrets out of both the model's context and any table the feature's role can reach. A system-prompt rule and a keyword blocklist are not fixes, because a rephrase or an encoding defeats them.
© 2026 GenAI Security Lab. All rights reserved. You may read, quote, and link to this material with attribution. Copying, republishing, redistribution, resale, or use to train models or build competing products is prohibited without prior written permission.