McDonald's ends AI drive-thru pilot after viral order errors at 100+ restaurants
McDonald's / IBM · Jun 17, 2024 · Customer service agent
What happened
After testing IBM's automated order taking at more than 100 US drive-thrus since 2021, McDonald's told franchisees the technology would be shut off no later than July 26, 2024. Viral videos showed the voice agent adding hundreds of dollars' worth of chicken nuggets to one order and bacon to another customer's ice cream. McDonald's said it still believes voice ordering is part of its restaurants' future.
Causal vector
Voice agent committed customers to orders it misheard, with no bound on order size or plausibility
Source
Reported by BBC News. Verified against the primary report.
An order is a transaction. A per-transaction cap sized to a plausible drive-thru basket returns BLOCKED on a runaway nuggets order at the point of sale — converting a viral, brand-damaging failure into a silent retry.
The six rule types that contain this class of failure
Per-transaction cap
Any single spend above your ceiling is BLOCKED before it moves.
Daily total
Cumulative spend across all agent calls, bounded per day.
Velocity limit
Stops runaway retry loops — the #1 cause of overnight losses.
Merchant allowlist
Only approved destinations can ever receive funds.
Category rules
Flag high-risk classes (crypto, infra, refunds) for review.
Approval threshold
Above a value, the action waits for a human.
Related incidents
Anthropic test agent buys a duplicate snowboard its owner already owns
Anthropic · Oct 1, 2025
OpenAI Operator AI agent makes surprise smoked-salmon purchase
OpenAI · Mar 9, 2025
OpenAI Operator agent makes unauthorized $31.43 grocery purchase
OpenAI · Feb 7, 2025
Don't be the next entry
Every incident in this database is the result of trusting a prompt, a provider cap, or a human review cycle. sipi.bot replaces all three with one deterministic call. 67 documented failures, one control.