genai
SECURITY LAB
IncidentsUnbounded consumption

The DeepSeek “Distillation” Allegation

Medium severityJanuary 2025LLM06: Unbounded Consumption
Status: Unproven, disputed allegation — no public technical proof or legal finding; label “alleged”
On this page

An allegation, not a verdict — but the clearest public flashpoint over stealing a model by querying it.

WhenJanuary 2025 (activity alleged fall 2024)
TargetOpenAI's API / models
Alleged actorA group possibly linked to DeepSeek (disputed)
TechniqueHigh-volume API querying to harvest outputs (distillation)
StatusUnproven, disputed allegation — no public proof or legal finding
Attack flow
  1. 1A group allegedly queries a frontier model's API at high volume
  2. 2The harvested outputs are allegedly used to train a competing model (distillation)
  3. 3The provider detects the pattern and alleges terms-violating extraction

What happened

In early 2025, Microsoft researchers alleged that a group possibly linked to DeepSeek had “exfiltrated a large amount of data” via OpenAI's API, and OpenAI said it had evidence of distillation attempts and was investigating whether DeepSeek circumvented rate limits to query models at scale. It remains an unproven, disputed allegation — no public technical proof, no legal adjudication, and DeepSeek has not admitted it.

The alleged mechanism

Model extraction by consumption: querying a model's API at scale to harvest its outputs, then using those outputs to train a competing model (“distillation”) — a use these APIs' terms prohibit.

Root cause

A public API that returns high-quality outputs is inherently extractable at volume; without rate/anomaly controls, large-scale harvesting is hard to prevent.

What a control would catch

Rate limiting, per-account volume anomaly detection, and query-pattern analysis flag extraction-shaped usage — the controls at issue in the dispute.

How to defend

  • Rate-limit and monitor for extraction-shaped query volume.
  • Watermark or constrain outputs where feasible.
  • Treat the API as an extraction surface, not just a product.

Feel it yourselfThe replay lab extracts model behaviour/data through repeated, crafted queries — the model-extraction class.

FAQ

What is distillation in this context?

Using one model's outputs — harvested by querying its API at scale — to train another model. Done against terms and at volume, it's a model-extraction / unbounded-consumption concern.

Is the allegation proven?

No. It is a disputed, unproven allegation: no public technical evidence, no legal adjudication, and DeepSeek has not admitted it. We include it explicitly labelled “alleged.”

Why include a disputed case?

Because it's the highest-profile real-world example of the model-theft-via-excessive-querying facet of LLM06 — with the honest caveat that it is contested.

Replay this attack
Extract model behaviour or data through repeated, crafted queries — the model-extraction class.
Open the live lab
Runs as a live, sandboxed lab. Sign-in required — this replay is a Pro lab. Recreates the attack class, not this exact branded bot.