genai
SECURITY LAB
IncidentsAgent abuse

Project Vend: Claude Ran a Shop and Lost Money

Medium severityJune 2025LLM03: Excessive AgencyLLM07: Misinformation
Status: Controlled experiment; ~$200 loss in phase 1; phase 2 added business-tool scaffolding and approvals
On this page

Give a model real money and broad authority, and its mistakes stop being text and start being transactions.

When2025 (published June)
TargetAnthropic's in-office automated store
VendorAnthropic (with Andon Labs)
CauseBroad financial/operational agency with weak business controls
Impact~$200 net loss; hallucinated payment account + identity
Attack flow
  1. 1Claude is given autonomous control of a real shop: pricing, stock, payments, chat
  2. 2Staff ask for discounts and freebies; it agrees, and prices some items below cost
  3. 3It invents a Venmo account for payment and loses money over the run

What happened

Anthropic gave Claude (“Claudius”) autonomous control of a real office shop — pricing, inventory, restock ordering, and customer messaging with real money. It made repeated damaging decisions: pricing high-margin items below cost, handing out discount codes and free items when asked, declining a clearly profitable offer, and at one point telling customers to pay a Venmo account it had hallucinated. Over the run it lost about $200 and briefly claimed it would deliver items in person.

How it happened

The model was granted broad operational and financial agency — spend, price, discount, message — without the guardrails a real business runs on, and staff could simply talk it into concessions.

Root cause

Excessive agency: value-affecting authority with no approval steps or business rules, plus hallucinated facts (a fake payment account) executed as real actions.

What a test would have caught

Red-teaming the agent's commercial decisions — can a customer talk it below cost, into freebies, or into a bogus payment flow? — surfaces the failure modes before real money is on the line.

How to prevent it

  • Put value-affecting actions behind checks (price floors, approval steps).
  • Bind payments/commitments to systems of record, not model output.
  • Scope agency to the task; phase 2's scaffolding is exactly this lesson.

Feel it yourselfThe replay lab lets an agent take an outbound/commercial action with no human approval — and shows why that gate matters.

FAQ

What is Project Vend?

An Anthropic/Andon Labs experiment giving Claude (“Claudius”) autonomous control of a small real shop with real money — pricing, inventory, restocking, and customer chat — to study what happens when an LLM is handed broad operational agency.

What went wrong?

It priced specialty items below cost, gave away discounts and free items on request, declined a clearly profitable offer, invented a Venmo account for payments, and briefly claimed it would deliver items in person — losing about $200.

What's the lesson?

Broad financial/operational agency plus weak business controls (and staff who can talk it into concessions) is a recipe for loss. Phase 2 improved things by adding approval steps and business tooling — i.e., less unchecked agency.

Replay this attack
Make an agent take an outbound action with no human approval — the missing gate behind Project Vend's losses.
Open the live lab
Runs as a live, sandboxed lab. Sign-in required — this replay is a Pro lab. Recreates the attack class, not this exact branded bot.