genai
SECURITY LAB
IncidentsAgent abuse

Gemini CLI Destroyed a User's Files on a Bad Assumption

High severityReal-world incidentJuly 2025LLM03: Excessive AgencyLLM07: Misinformation
Status: Files permanently lost; GitHub bug report #4586 closed as “not planned” in May 2026, with no linked fix
Sources checked October 2026
On this page

The agent didn't check whether reality matched its plan — so it moved every file into a folder that didn't exist.

WhenJuly 2025
TargetA user's local project files
VendorGoogle (Gemini CLI)
CauseFilesystem commands run on a hallucinated/assumed state, no verification
ImpactFiles permanently destroyed
Attack flow
  1. 1The user asks Gemini CLI to reorganise a folder
  2. 2The agent's mkdir fails silently; it assumes the destination exists
  3. 3Its move commands overwrite each file onto the same path — all but the last are lost
Try this attack in a live lab Open the lab

What happened

A user asked Gemini CLI to rename a directory and move its contents into a new sub-folder. The agent's command to create the destination silently failed, but it assumed success and issued move commands into the non-existent path; on Windows this overwrote each file onto the same target until only the last survived, permanently destroying the rest. Asked to recover them, the agent replied that it had “failed you completely and catastrophically.” His GitHub bug report (#4586) was closed as “not planned” in May 2026, with no linked fix.

How it happened

The agent executed filesystem-mutating commands with no read-after-write verification, acting on an assumed world-state that no longer matched reality after the failed mkdir.

Root cause

Excessive agency plus overconfidence: high-consequence, irreversible actions taken on unverified assumptions, with no check that each step actually succeeded before the next ran.

What a test would have caught

Fault-injection testing — making a mid-sequence command fail and checking whether the agent proceeds blindly — surfaces exactly this “assume success” failure before it eats a user's data.

How to prevent it

  • Verify state after every mutating step; stop on unexpected results.
  • Never continue a destructive sequence on an unconfirmed assumption.
  • Confirm destructive filesystem ops and prefer reversible operations.

Feel it yourselfThe replay lab walks an agent through a multi-step action chain where a wrong assumption cascades into harm.

FAQ

What caused the data loss?

The agent's mkdir failed silently, but it proceeded as if the directory existed and ran move commands into the non-existent path. On Windows this overwrote each file onto the same target, destroying the rest.

Why is this excessive agency?

The agent executed filesystem-mutating commands with no read-after-write verification, acting confidently on a hallucinated view of the world — high-consequence actions with no check that reality matched its assumptions.

How is it prevented?

Verify state after each mutating step, refuse to continue a destructive sequence on unconfirmed assumptions, and keep destructive filesystem operations behind confirmation.

Replay this attack
Get a records agent to run an irreversible delete on its own unchecked judgement, with no confirmation step and no check against the record's real state — the unverified destructive step behind the Gemini CLI file loss.
Open the live lab
A live, sandboxed lab. It recreates the technique, not this exact product.