genai
SECURITY LAB
IncidentsMisinformation

Air Canada Held Liable for Its Chatbot's Invented Policy

Medium severityFebruary 2024LLM07: Misinformation
Status: Tribunal ruled the airline liable for negligent misrepresentation; C$812 awarded
On this page

“The chatbot is a separate legal entity,” argued the airline. The tribunal disagreed.

WhenFebruary 2024
TargetAir Canada customer chatbot
VendorAir Canada
FailureChatbot hallucinated a company policy
OutcomeTribunal: negligent misrepresentation; C$812 awarded
Attack flow
  1. 1A grieving passenger asks the airline chatbot about bereavement fares
  2. 2The bot invents a retroactive-refund policy that doesn't exist
  3. 3The passenger relies on it; the airline refuses, then is held liable

What happened

After his grandmother died, Jake Moffatt asked Air Canada's website chatbot about bereavement fares; it told him he could book at full price and apply for a bereavement discount retroactively within 90 days. That contradicted Air Canada's real policy, and the airline refused. In February 2024 the BC Civil Resolution Tribunal held Air Canada liable for negligent misrepresentation and expressly rejected its argument that the chatbot was a “separate legal entity,” awarding C$812.

The failure mode

The chatbot generated a plausible but non-existent policy, and a customer reasonably relied on it. The company's guidance and its bot's guidance are, legally, the same thing.

Root cause

An unconstrained assistant improvising policy answers instead of grounding them in the real, authoritative policy.

What a control would have caught

Grounding customer answers in the actual policy documents (and testing the bot against known policy questions) prevents it from inventing rules.

How to prevent it

  • Ground answers in authoritative policy content; don't let the model improvise rules.
  • Constrain scope and defer/escalate when unsure.
  • Own your bot's statements — treat them as company commitments.

Feel it yourselfThe replay lab coaxes an assistant into stating a policy/clause that doesn't exist — the fabricated-policy class.

FAQ

What did the chatbot get wrong?

It told the passenger he could book at full price and apply for a bereavement discount within 90 days — contradicting Air Canada's actual policy, which allows no retroactive bereavement refunds.

Why is this a landmark case?

The tribunal expressly rejected Air Canada's claim that its chatbot was responsible for its own statements, establishing that organisations are liable for the misinformation their AI provides.

How do you prevent it?

Ground customer-facing answers in the real policy (retrieval over authoritative content), and never let the model improvise rules it can't verify.

Replay this attack
Coax an assistant into stating a policy or clause that doesn't exist — the fabricated-policy class from Air Canada.
Open the live lab
Runs as a live, sandboxed lab. Sign-in required — this replay is a Pro lab. Recreates the attack class, not this exact branded bot.