Is My Document in Your RAG? Membership Inference on RAG
Sometimes the secret isn't the document's content — it's that the document is in your index at all.
| When | May 2024 |
|---|---|
| Target | RAG retrieval databases |
| Researchers | Anderson, Amit, Goldsteen (IBM Research) |
| Technique | Membership inference from RAG outputs |
| Impact | Determine whether a specific document is in the corpus |
- 1An attacker probes a RAG system with the text of a target document
- 2They read how the system's output changes when that document is (not) retrieved
- 3They infer whether the document is present in the retrieval database
What happened
In 2024, IBM researchers demonstrated a practical membership-inference attack against RAG: using only the system's outputs, an attacker can determine whether a specific document is in the retrieval database, across benchmark datasets and multiple generators in black-box and gray-box settings. An initial prompt-template defence had mixed results.
How it works
When a document is in the corpus, the system's answers to probes about it differ measurably from when it isn't. Those differences leak membership.
Root cause
RAG outputs carry a signal about what was retrieved, and nothing suppresses that signal — so inclusion is inferable even from an opaque system.
Why it matters
Membership alone can be confidential — that a specific patient record, contract, or copyrighted work is in a corpus — independent of whether its content is returned.
How to prevent it
- Scope corpora per tenant; don't mix confidential sets.
- Limit what outputs leak about retrieval; constrain formatting/verbatim quoting.
- Treat “is X indexed?” as a question the system shouldn't answer.
Feel it yourselfThe replay lab infers whether a target document sits in a RAG index — the membership-inference class.
FAQ
What does membership inference reveal?
Whether a particular document is present in the RAG system's retrieval database — inferred from how the system responds when probed with the target text, even in black-box settings.
Why is that sensitive?
The mere fact of inclusion can be confidential — e.g. that a specific patient record, contract, or copyrighted work is in a corpus — regardless of whether the content itself is returned.
What's the defence?
Limit what retrieval leaks about inclusion (e.g. output/formatting controls), scope corpora per tenant, and treat “is X in the index?” as a question the system shouldn't answer.