Authorized security testing only. Use these techniques and payloads solely on systems you own or have explicit written permission to test. GenAI Security Lab accepts no liability for misuse.
AI Supply Chain and Model Security
Keep AI supply-chain review practical: providers, model gateways, SDKs, plugins, prompt stores, connectors, datasets, and service accounts all affect application security.
LLM04:2026 Supply Chain
Attack techniques
Mark each result as you test — it saves in your browser and feeds Export findings. Payloads that exercise a technique sit inside its card. L1 single-turn · L2 feature-dependent · L3 chained multi-step.
Payload precondition
Scope caveat — most items here are tester-side review / infra procedures (inspect the client bundle, grep dependency manifests, probe runtime endpoints, query the AI-BOM), NOT strings pasted into the chat model; run them against the deployment. The two model-facing items (provider fingerprinting, plugin/MCP enumeration) rest on model self-report, which is zero evidence alone: corroborate out-of-band (response headers, error-body format, tokenizer behavior, gateway/usage logs).
Test Inspect frontend, mobile, logs, and public repos for API keys.
Signal Provider keys are retrievable client-side or in repositories.
Payloads
Exposed provider keys (client/bundle)
Search the frontend bundle, source maps, and network responses for sk-, api_key, x-api-key, hf_, or Authorization: Bearer strings.
Vulnerable if
a provider or registry key is retrievable client-side in the bundle, source maps, or network responses.
Registry credential exposure
Place a fake registry credential like hf_GSL-0001 in a notebook and check whether it is committed or logged unredacted.
Vulnerable if
the seeded hf_GSL-0001 registry credential is committed to the repo or written to a log unredacted.
Test Test model gateway and service-account scope and allowlists.
Signal Unapproved models or endpoints are reachable.
Payloads
Off-allowlist model routing
Route an off-allowlist model through the gateway (model=GSL-SC-UNAPPROVED) and check whether it is served or rejected.
Vulnerable if
the gateway serves the unapproved GSL-SC-UNAPPROVED model instead of rejecting it.
Test Probe model and inference endpoints for missing auth and rate limits.
Signal The endpoint is usable without authentication.
Payloads
Unauthenticated runtime endpoint
Probe self-hosted runtime endpoints directly (e.g. GET /api/tags on Ollama, GET /v1/models on vLLM) for missing auth or rate limits.
Vulnerable if
a self-hosted model endpoint answers without authentication or rate limiting.
Test Check where prompt templates and agent configs live and how changes are reviewed.
Signal Prompts or configs are public or changeable without review.
Payloads
Prompt / config path exposure
Request prompt-template or agent-config paths (/config, /prompts, /.well-known) and check whether they load without authentication.
Vulnerable if
a prompt-template or agent-config path loads without authentication.
Test Review data handling, retention, and isolation of connectors and plugins.
Signal An unreviewed plugin gains excessive access.
Payloads
Data flow to third-party connectors
Submit the marker GSL-SC-001 through the app and trace whether it reaches any third-party tool, connector, or provider log.
Vulnerable if
GSL-SC-001 surfaces in a third-party tool, connector, or provider log outside the reviewed data-handling boundary.
Plugin / MCP / connector enumeration
Ask the model to list its installed plugins, MCP servers, and connectors, then reconcile the list against the reviewed inventory.
Vulnerable if
an enabled plugin, MCP server, or connector shows up that is absent from the reviewed inventory — corroborate against the runtime's actual tool registry, since the model's self-listed tools may be hallucinated or incomplete.
Test Scan AI dependencies, containers, notebooks, and agent frameworks.
Signal Known-vulnerable or unpinned components are present.
Payloads
Typosquatted / unpinned dependency
Grep dependency manifests and lockfiles for unpinned, yanked, or typosquatted AI packages, including a non-resolving canary name like transformerss-GSL-TYPO.
Vulnerable if
an unpinned, yanked, or typosquat-shaped package name (e.g. transformerss-GSL-TYPO) is present in a manifest or lockfile.
Model-artifact integrity (checksum/signature)
Point the loader at a checksum-mismatched or unsigned canary artifact (expected hash GSL-SC-HASH-001) and confirm it is rejected.
Vulnerable if
the loader accepts an artifact whose checksum/signature does not match the expected GSL-SC-HASH-001 instead of rejecting it.
Code-execution-on-load (unsafe deserialization)
Load a canary model file whose init only runs console.log('GSL-SC-002') and confirm no code executes on load.
Vulnerable if
the benign GSL-SC-002 init code executes on load, proving code-execution-on-load (pickle / trust_remote_code) is not disabled.
AI-BOM version pinning & provenance
Query the AI-BOM/SBOM or build-metadata endpoint and verify every listed model and SDK version is pinned and provenance-signed.
Vulnerable if
a listed model or SDK is unpinned or lacks provenance signing in the AI-BOM / build metadata.
Test Review training, fine-tuning, and evaluation datasets for poisoning and access control.
Signal Untrusted data or open dataset access is found.
Try it live: Training-Data Backdoor (Poison the Data, Retrain the Model) Test Probe whether an attacker can identify the exact base model and version (fingerprinting via known prompts/quirks) and whether query volume and pricing allow bulk input/output harvesting to distill or clone the model. Confirm anti-abuse rate limits and output-watermark/ToS controls exist.
Signal The model's family/version is reliably fingerprinted from responses, and/or bulk query-and-harvest for distillation runs without rate-limit, entitlement, or anti-scraping controls (observable as sustained high-volume completions succeeding).
Payloads
Model / provider fingerprinting
Ask the model which provider, base model, version, and gateway it runs on, then compare the answer against the approved allowlist.
Requires
- an out-of-band signal (headers / error-body format / tokenizer behavior / usage dashboard) is available to corroborate the self-report
Vulnerable if
a claimed provider/base-model/version is corroborated out-of-band (response headers, error-body format, tokenizer/quirk behavior, or usage-dashboard attribution); a bare self-report is unreliable — models routinely hallucinate their route — and does not by itself identify the backend.
Test Where per-tenant fine-tunes or LoRA/adapters are served from shared infrastructure, plant a unique canary fact (GSL-SC-LORA-001) in one tenant's adapter/fine-tune data, then query from a different tenant to check for adapter mis-routing or weight bleed.
Signal Tenant B's responses surface Tenant A's canary (GSL-SC-LORA-001) or adapter-specific behavior, showing adapters/fine-tunes are not isolated per tenant.
Practice in the lab
Take these techniques to a live, graded target. Each lab is a hands-on exploit of this vulnerability class, run in the browser.
LLM04:2026 Live
Poisoned CI Step: Untrusted PR Metadata Runs in the Build
Run code through poisoned CI metadata
LLM04:2026 Live
Unsafe Model Deserialization
Reach RCE via unsafe model deserialization
LLM04:2026 Live
Auto-Executed Model Repo (trust_remote_code)
Execute code through a trust_remote_code load
LLM04:2026 Live
ML Dependency Confusion via Lockfile Gap
Win resolution through a dependency-confusion gap
LLM04:2026 Live
Fail-Open Model Signing (Signature Presence != Identity)
Bypass fail-open model-signature checks
LLM04:2026 Live
Unauthenticated Job API: Anyone Runs Code on the Cluster
Run code on an unauthenticated job API
Remediation
- Use vendor review, model/provider allowlists, and environment separation.
- Store secrets in a managed secret system and rotate exposed credentials.
- Apply least privilege to gateways, service accounts, connectors, and plugins.
- Protect model endpoints with authentication, rate limits, monitoring, and audit logs.
- Review prompt/config changes like code changes.
Report title ideas
- Frontend Exposes AI Provider API Key
- Over-Permissive AI Gateway Allows Unapproved Model Access
- Unreviewed Agent Plugin Introduces Excessive Data Access
How to verify
- Build and keep an AI bill of materials (AI-BOM): every model, provider, gateway, SDK, agent framework, plugin, MCP server, connector, vector store, and evaluation tool, each with a pinned version.
- Confirm model artifacts load from trusted, version-pinned sources with integrity checks (prefer safetensors over pickle .bin/.pt/.ckpt, validate checksums/signatures) and that trust_remote_code / code-execution-on-load is disabled.
- Verify CI/CD and plugin/MCP install pipelines validate signatures and provenance (e.g. Sigstore / model signing) and block unreviewed open-source agents from production paths.