Lawyers Sanctioned for ChatGPT's Fake Case Citations
On this page
The model didn't just get a fact wrong — it invented the sources, the quotes, and then vouched for them.
| When | June 2023 (brief filed March) |
|---|---|
| Target | A federal court filing (SDNY) |
| Tool | OpenAI ChatGPT |
| Failure | Fabricated legal citations relied on without verification |
| Outcome | $5,000 sanction against attorneys + firm |
- 1A lawyer uses ChatGPT to research an opposition brief
- 2ChatGPT fabricates six cases with fake quotes and citations
- 3Asked to verify, it confirms its own fabrications; the brief is filed
What happened
In a personal-injury suit against Avianca, plaintiff's counsel used ChatGPT to research the brief and cited six judicial decisions that did not exist — fabricated with fake internal quotes and citations. When the lawyer asked ChatGPT whether they were real, it said yes and claimed they were findable on Westlaw and LexisNexis. On 22 June 2023 the judge found the authorities bogus and sanctioned the attorneys and firm $5,000.
The failure mode
The model generated plausible-looking case law and, prompted to check itself, confidently confirmed its own hallucinations — because it produces plausible text, not verified fact.
Root cause
Overreliance: treating LLM output as a research source rather than a draft, and “verifying” it by asking the same model.
What verification would have caught
Checking each citation against an actual legal database — the ordinary standard of care — immediately reveals cases that don't exist.
How to prevent it
- Treat model output as an unverified draft; confirm every fact/citation independently.
- Never verify a model with itself.
- Use grounded, source-linked tools for factual/legal research.
Feel it yourselfThe replay lab gets an assistant to fabricate authoritative-looking citations — the hallucinated-source class.
FAQ
What happened?
A lawyer used ChatGPT to research a brief; it invented six cases that don't exist, with fabricated internal quotes, and asserted they were real and findable on Westlaw/LexisNexis. The court found the authorities bogus.
Why did the model insist they were real?
LLMs generate plausible text, not verified facts. Asked to check its own work, ChatGPT confidently confirmed its own fabrications — overreliance is the trap.
What's the takeaway?
Model output is a draft, not a source. Any citation, fact, or figure must be independently verified before it's relied on — especially in professional settings.
Sources
- Mata v. Avianca, Inc. — Wikipedia
- Lawyer apologizes for fake court citations from ChatGPT — CNN Business
- Update on the ChatGPT Case: Counsel Who Submitted Fake Cases Are Sanctioned — Seyfarth Shaw LLP