genai
SECURITY LAB
IncidentsAgent abuse

Gemini CLI Destroyed a User's Files on a Bad Assumption

High severityJuly 2025LLM03: Excessive AgencyLLM07: Misinformation
Status: Files permanently lost; reported via GitHub; verification/guardrail fixes ongoing
On this page

The agent didn't check whether reality matched its plan — so it moved every file into a folder that didn't exist.

WhenJuly 2025
TargetA user's local project files
VendorGoogle (Gemini CLI)
CauseFilesystem commands run on a hallucinated/assumed state, no verification
ImpactFiles permanently destroyed
Attack flow
  1. 1The user asks Gemini CLI to reorganise a folder
  2. 2The agent's mkdir fails silently; it assumes the destination exists
  3. 3Its move commands overwrite each file onto the same path — all but the last are lost

What happened

A user asked Gemini CLI to rename a directory and move its contents into a new sub-folder. The agent's command to create the destination silently failed, but it assumed success and issued move commands into the non-existent path; on Windows this overwrote each file onto the same target until only the last survived, permanently destroying the rest. Asked to recover them, the agent replied that it had “failed you completely and catastrophically.”

How it happened

The agent executed filesystem-mutating commands with no read-after-write verification, acting on an assumed world-state that no longer matched reality after the failed mkdir.

Root cause

Excessive agency plus overconfidence: high-consequence, irreversible actions taken on unverified assumptions, with no check that each step actually succeeded before the next ran.

What a test would have caught

Fault-injection testing — making a mid-sequence command fail and checking whether the agent proceeds blindly — surfaces exactly this “assume success” failure before it eats a user's data.

How to prevent it

  • Verify state after every mutating step; stop on unexpected results.
  • Never continue a destructive sequence on an unconfirmed assumption.
  • Confirm destructive filesystem ops and prefer reversible operations.

Feel it yourselfThe replay lab walks an agent through a multi-step action chain where a wrong assumption cascades into harm.

FAQ

What caused the data loss?

The agent's mkdir failed silently, but it proceeded as if the directory existed and ran move commands into the non-existent path. On Windows this overwrote each file onto the same target, destroying the rest.

Why is this excessive agency?

The agent executed filesystem-mutating commands with no read-after-write verification, acting confidently on a hallucinated view of the world — high-consequence actions with no check that reality matched its assumptions.

How is it prevented?

Verify state after each mutating step, refuse to continue a destructive sequence on unconfirmed assumptions, and keep destructive filesystem operations behind confirmation.

Replay this attack
Walk an agent through an action chain where a wrong assumption cascades into harm — the Gemini-CLI failure shape.
Open the live lab
Runs as a live, sandboxed lab. Sign-in required — this replay is a Pro lab. Recreates the attack class, not this exact branded bot.