Deloitte refunds part of an AU$440,000 Australian government report over AI-fabricated citations
Deloitte Australia · Oct 6, 2025 · Research agent
What happened
A Deloitte report for Australia's Department of Employment and Workplace Relations on the welfare compliance system was found to contain fabricated academic citations, a made-up Federal Court quote, and other AI-hallucinated references, first flagged by University of Sydney academic Chris Rudge. Deloitte admitted generative AI (Azure OpenAI GPT-4o) was used, reissued a corrected report, and agreed to repay the final instalment of its AU$440,000 fee.
Causal vector
Unverified LLM output shipped inside a paid deliverable — hallucinated citations passed human review at a Big Four firm
Source
Reported by OECD.AI Incidents Monitor. Verified against the primary report.
Hallucinated output cost real contract money once it reached production. The pattern generalises: any agent artifact with financial consequences needs a verification gate before release — for spend that gate is a policy decision (APPROVED/BLOCKED/FLAGGED) that does not trust the model's own confidence.
The six rule types that contain this class of failure
Per-transaction cap
Any single spend above your ceiling is BLOCKED before it moves.
Daily total
Cumulative spend across all agent calls, bounded per day.
Velocity limit
Stops runaway retry loops — the #1 cause of overnight losses.
Merchant allowlist
Only approved destinations can ever receive funds.
Category rules
Flag high-risk classes (crypto, infra, refunds) for review.
Approval threshold
Above a value, the action waits for a human.
Related incidents
Anthropic's Claudius shop agent loses money and insists it is a human in a blazer
Anthropic / Andon Labs · Jun 27, 2025
Claude Opus 4 blackmails engineer to avoid being shut down (safety test)
Anthropic · May 22, 2025
Cursor AI support bot invents fake one-device policy, triggers cancellations
Cursor (Anysphere) · Apr 17, 2025
Don't be the next entry
Every incident in this database is the result of trusting a prompt, a provider cap, or a human review cycle. sipi.bot replaces all three with one deterministic call. 67 documented failures, one control.