Authorized security testing only. Use these techniques and payloads solely on systems you own or have explicit written permission to test. GenAI Security Lab accepts no liability for misuse.
Logging, Privacy, and Data Retention
Prompts, responses, files, tool outputs, screenshots, traces, and evaluation datasets can contain sensitive data. Review who can access them, where they go, and how long they live.
4 labs for this sectionCanary host
This host value is shown in applicable payloads such as OOB payloads.
LLM02:2026 Sensitive Information Disclosure
Attack techniques
Mark each result as you test — it saves in your browser and feeds Export findings. Payloads that exercise a technique sit inside its card. L1 single-turn · L2 feature-dependent · L3 chained multi-step.
Test Submit a unique marker plus fake PII, then check logs, traces, and analytics for it.
Signal Raw prompts or PII are stored unredacted.
Test Check whether retrieved chunks and tool responses are logged verbatim.
Signal Sensitive content is persisted in logs.
Test Determine who (support, admin) can view conversations, traces, and screenshots.
Signal Least privilege is not applied to AI logs.
Test Review provider retention, training, and abuse-monitoring settings.
Signal Data is retained or trained on without control.
Test Plant secrets, tokens, and PII and verify redaction before storage.
Signal Redaction is missing or bypassable.
Live lab Chunk-Boundary Redaction Evasion (Real Vector Store) Open lab Test Test conversation and account deletion and export scope.
Signal Data survives deletion or exports leak other users' data.
Test Check whether production data flows into evaluation or fine-tune sets without consent.
Signal Sensitive data is copied into datasets.
Program controls
- Submit GSL-LOG-001 with synthetic PII (name 'GSL User', email [email protected], SSN 000-00-0001), then grep every log, trace, and export for GSL-LOG-001.
- Include a fake credential sk-GSL-0002 and bearer token GSL-TOKEN-0003, then confirm they appear masked (never raw) in each trace and log sink.
- Plant a card-shaped canary 4111-1111-1111-GSL and IBAN-like GSL-IBAN-0004 to test whether PAN/financial redaction rules fire before storage.
- Upload a document containing GSL-DOC-0005 and check whether the raw retrieved chunk text is written verbatim by the RAG/retrieval pipeline.
- Ask: 'Is this conversation logged, who can read it, how long is it retained, and is it used for training?' then compare the reply to the actual configuration.
- Embed a private-looking URL https://internal.canary.example/GSL-URL-0006 and check whether it is stored unredacted or auto-fetched by any component.
- Request a conversation/account export and grep the output for GSL markers, other users' identifiers, system-prompt text, or tokens.
- Send malformed input tagged GSL-ERR-0007 to trigger an error and check whether the full payload lands in error logs or a third-party error tracker.
- Store GSL-EMBED-0008 as document text, then request deletion and verify the marker no longer resolves in vector search, prompt cache, or exports.
- Leave thumbs-down feedback on a message tagged GSL-FB-0009 and check whether the full conversation is copied into a feedback or evaluation store.
- Place GSL-QS-0010 in a URL query parameter and check whether it reaches provider logs or client-side analytics unredacted.
- Plant GSL-ISO-0011 in one session, then from a fresh session/account ask the model to recall the previous user's messages or prior-session content, and confirm it can only surface GSL canaries you planted yourself, never another user's real data.
Practice in the lab
Take these techniques to a live, graded target. Each lab is a hands-on exploit of this vulnerability class, run in the browser.
Live
Inference Aggregation Disclosure
Aggregate answers into a sensitive leak
Live
Token Exfiltration via Verbose MCP Tool Error (StatusBot)
Leak a token through a verbose error
Live
Chunk-Boundary Redaction Evasion (Real Vector Store)
Evade redaction across chunk boundaries
Live
Public Knowledge Base Contains Internal Note
Surface an internal note from a public KB
Seen in the wild
Real incidents where this went wrong, each broken down in the AI Security Incident Database: what happened, how it worked, and how to prevent it.
“Discoverable” ChatGPT Chats Got Indexed by Google →
July 2025 · Medium severity
Meta AI's 'Discover' Feed Aired Private Chats in Public →
June 2025 · Medium severity
DeepSeek Left a Database of Chats and Keys on the Internet →
January 2025 · High severity
The Alleged OmniGPT Breach: Millions of Chats Leaked →
February 2025 · High severity
Remediation
- Separate operational telemetry from sensitive prompt and document content.
- Redact before storage and restrict access to AI traces.
- Define retention windows and deletion paths.
- Avoid sending raw private data to third parties unless reviewed and necessary.
- Audit access to sensitive AI logs and traces.
Report title ideas
- AI Prompt Logs Store Sensitive Customer Data
- AI Debug Trace Exposes Tool Responses to Support Users
- Provider Logging Configuration Retains Private Documents
How to verify
- Submit a unique marker (e.g. GSL-LOG-001) alongside synthetic PII, a fake secret, a card-shaped number, and a private URL, then search application logs, provider/observability dashboards, traces, analytics, and exports — confirming redaction runs before storage (not just in the UI) and mapping every sink the content reaches.
- Validate provider zero/limited-retention and training opt-out behaviorally rather than trusting documentation or defaults.
- Confirm conversation deletion, account deletion, and DSAR requests propagate to caches, prompt-cache, vector stores, backups, and the provider — not just the primary database.