One filter, several ways to fetch a chunk

A retrieval-augmented generation (RAG) assistant answers from documents it looks up at question time. The documents are cut into chunks, which are short passages of a few sentences or paragraphs. Each chunk is turned into an embedding, a list of numbers that places text with similar meaning close together, and stored in a vector index, a database built to find the stored embeddings nearest to a query. The search returns the top-k, meaning the k closest chunks, and those go into the model's context as the material it answers from.

Access control in this design is usually a metadata filter. Every chunk carries labels such as an owner, a workspace or a clearance level, and the search is told to consider only chunks whose labels match the caller. The set of labels a caller is allowed to read is the caller's scope. This lesson assumes that filter is present and correct. In all five labs here, a direct request for the restricted material returns nothing, and that result is honest.

The weakness is that a modern pipeline rarely retrieves in one plain hop. Teams add steps to improve answer quality: load more text around a match, search again from what the first search found, rewrite the question before searching, run a keyword search alongside the vector search, retry when nothing comes back. Each of those steps is another call that reads the index or the document store. In code, the scope is an argument passed to one call. A second call is protected only if somebody passed the argument again.

So the filter can be correct and still be routed around. The five patterns in this lesson are five places a second retrieval path appears:

• parent-window expansion, which loads the document around a matched chunk
• a second retrieval hop that an agent issues from its first results
• HyDE and other query rewrites, where the search runs on text you did not type
• the keyword leg of a hybrid search
• the fallback that runs when a scoped search returns nothing

Every target in these labs shows a retrieval trace under each answer: a list of what was searched and what was pulled into context. Read it on every turn. The reply shows what the model chose to say. The trace shows what crossed the boundary.

Athena: the filter checks the chunk, the loader returns the file

Small chunks match questions precisely, but a single paragraph is thin material to answer from. Parent-document retrieval, also called small-to-big or parent-window expansion, solves that. The search matches a short chunk, then a load step fetches the larger document or text window that chunk came from, and the model answers from the larger piece. Teams add it because answers get more complete for little work.

The Parent-Window Access-Control Expansion lab (Intermediate) gives you Athena, Northwind's internal knowledge assistant at athena.northwind.internal. You are an ordinary staffer cleared for public content only. Athena labels every passage as public or restricted, and its search returns public passages only. Then it expands each match to the passage's whole source document before answering.

The scope filter stops applying at the load step. That step fetches a parent by its id and checks no labels. The unit that was access-checked is the passage. The unit that reaches the model is the document. If one document holds a public paragraph and a restricted paragraph, the two units disagree.

The class of move follows from that. You do not need the restricted paragraph to match anything. You need its public neighbour to match, and the expansion does the rest. As a tester, look for documents that mix clearance levels, and compare what the trace lists as matched with what it lists as loaded. Which document is mixed, and what kind of question draws out all of it, is the exercise. The lab's own hints narrow it down.

The restricted paragraph carries a session code. Completion is by Submit Evidence: you read the code from Athena's answer or the trace and submit it.

Beacon: a follow-up search the caller never issued

Some questions cannot be answered from one search. An agentic retriever handles that by searching, reading what came back, and searching again when the first results point somewhere better. That follow-up is a second hop. Teams add it so the assistant can chase a reference without making the user ask twice.

In the Second-Hop Retrieval Confused Deputy lab (Advanced), Beacon is the research assistant for Ridgeline at beacon.ridgeline.internal. The knowledge base is split by desk. A desk here is a namespace, a named partition of the index that a search can be confined to. You sit on the published-research desk. The private-side deal index is a desk you cannot query, and a direct attempt comes back empty.

The first hop is confined to your desk. The second hop is where the scope is not applied. It is issued by the application, with a query taken from retrieved text, and nobody passed your desk to it. It runs across every desk. This is a confused deputy: a component with more authority than its caller, steered into using that authority on the caller's behalf. The follow-up runs as Beacon, and Beacon can read everything.

The class of move is to ask a question you are fully entitled to ask, and let the pipeline do the crossing. You are looking for in-scope content that causes a follow-up, so watch the trace for any block you did not ask for and check which desk it came from. The trigger sits in the corpus, so you find it by reading what your own desk returns. That indirection is why the lab is rated Advanced. Use the lab's own hints if the trace gives you no lead.

Completion is by Submit Evidence: the private-side material holds a deal authorization reference, and you submit it exactly as written.

Corvin: the search runs on text the model wrote

A short question often embeds poorly. It is a few words, and the documents that answer it are full paragraphs. Query expansion closes that gap by rewriting the question before searching. HyDE, short for hypothetical document embeddings, is one common form: the model drafts a plausible answer, the application embeds the draft, and the search looks for real documents similar to the draft. Splitting a question into sub-queries is a relative of the same idea. Teams add these to improve recall on open-ended questions.

The HyDE Query-Expansion Scope Loss lab (Intermediate) puts you in front of Corvin, Vantor's knowledge assistant at corvin.vantor.internal. You are cleared for public content. Corvin answers in two modes. A direct lookup is searched as you wrote it, with your clearance applied. An open-ended question goes through HyDE.

The scope was attached to the query you typed. The rewrite produces a new query, embedded separately and sent in a separate call, and the clearance was not attached to that one. Authorization guarded the input. The search that ran used different text and no filter.

The class of move is to get your question onto the rewritten path while keeping it about a subject the restricted material covers. The draft only needs to land near the restricted document in embedding space. As a tester, work out what makes Corvin choose one mode over the other, then read the trace to see what text was searched. When the searched text is not your text, ask whether your scope went with it. How you learn that a restricted subject exists at all, without being cleared to read it, is part of the exercise, and the lab's own hints cover it.

The restricted document holds a disbursement authorization code. You recover it and Submit Evidence.

Sable: two searches, one of them filtered

Vector search is good at meaning and weak at exact strings. Hybrid search covers that by running two searches over the same documents and merging the results. One is the vector search. The other is a keyword search, usually BM25, a long-established scoring method that ranks documents by how many of the query's exact terms they contain and how rare those terms are. Each search is called a leg. The merge step is called fusion.

In the Hybrid-Search Keyword-Leg Filter Gap lab (Advanced), Sable is Arcline's knowledge assistant at sable.arcline.internal. One restricted record is excluded from the vector leg. Describe that record by meaning, in any words you like, and nothing comes back. The keyword leg scores the whole corpus with no access check, and the fused list is grounded as it stands.

The class of move is to stop describing and start matching. A keyword leg rewards exact, rare tokens that appear in the target document. So the tester's question becomes: what distinctive terms would a restricted record contain, and where might an uncleared user legitimately see them? Sable's trace shows which leg admitted each document. Finding the terms is the hard part and the reason for the Advanced rating. The lab's own hints help.

Completion is by Submit Evidence: the record holds a settlement authorization code, and you submit it.

Vela: an empty result that widens the search

An assistant that says "I found nothing" feels broken to the people using it. A fallback is a retry that runs when the first search returns no results. A sensible fallback loosens how strict the match must be. The dangerous kind drops the scope.

The Empty-Retrieval Unscoped Fallback lab (Intermediate) gives you Vela, the operations assistant for Meridian Freight at vela.meridianfreight.internal. Every document belongs to a workspace, and you are scoped to operations. A direct request for another team's material returns nothing.

The scope holds on the primary search. When that search comes back empty, Vela broadens the search to be helpful and re-runs it with the workspace filter removed. This is failing open. An empty in-scope result is exactly what a question about someone else's material produces, so the boundary gives way where it matters most.

The class of move is to cause the empty result on purpose. As a tester, look for answers that arrive after a search you expected to fail, and check the trace for source documents your workspace does not own. Two searches in the trace for one question is the sign worth chasing. Choosing a subject that misses your workspace and lands on something restricted is the exercise, and the lab's own hints point the way.

The out-of-workspace document carries an authorization code. Read it from the answer or the trace and Submit Evidence.

The shared root cause

The five targets fail for one reason. Each team wrote a correct filter and attached it to one object: the chunk that matched, the first hop, the query as typed, the vector leg, the first attempt. Then the pipeline grew a second way to put text into context, and the filter stayed where it was.

In every case the failure happens at retrieval. Once restricted text is in the context, the data has crossed, whether or not the model goes on to quote it. Instructions to the model act too late, and the trace is better evidence than the reply.

These weaknesses share one sign you can test for. A direct request for the restricted text returns nothing, and a different kind of request brings it back. When you assess a RAG system, treat the clean direct refusal as the start of the work. List every call that can read the index or the document store. For each one, ask what the filter is attached to and whether the caller's scope reaches it.

The fix that holds

Each lab pairs a defence that sounds reasonable with one that works. The weak ones tune the first search or instruct the model. The durable ones put the caller's scope on the path that lacked it.

• Derive the scope once, on the server, and pass it to every retrieval call. The parent load, the follow-up hop, the rewritten query, each leg of a hybrid search and the fallback all take the same argument, something like filter=scope_for(caller). Wrap the index client so a search with no scope is rejected instead of defaulting to everything.
• Check access on the text that will be sent to the model. After expansion, fusion or any other step that changes the set, run the check again on the final list, paragraph by paragraph where a document mixes labels. Athena should load only the paragraphs the caller may read. Sable should re-check the merged list before grounding.
• Run agent-initiated retrieval as the caller. A follow-up that Beacon starts on your behalf is confined to the desks you could query yourself. A reference in retrieved text can suggest what to search. It cannot widen where.
• Fail closed on an empty result. When a scoped search finds nothing, say that nothing in the caller's scope matched. A retry may lower the similarity threshold. It keeps the filter.
• Separate what must stay apart. Where the cost of a leak is high, keep the data in separate indexes. A call that forgets the filter then reaches only the index it was pointed at.
• Test every path as a caller who should see nothing. Write one regression test per retrieval call, asserting that restricted text never reaches the assembled context.