Cursor AI support bot invents fake one-device policy, triggers cancellations
Cursor (Anysphere) · Apr 17, 2025 · Customer service agent
What happened
Cursor's AI support agent 'Sam' fabricated a policy claiming that 'Cursor is designed to work with one device per subscription as a core security feature' - a rule that didn't exist and actually stemmed from a backend bug. Enraged developers cancelled accounts; the company later clarified and apologized.
Causal vector
AI support agent confabulated a fake policy and enforced it against real users
Source
Reported by Ars Technica. Verified against the primary report.
Any agent action that changes a customer's entitlement (access revocation, plan change) should be a FLAGGED transaction requiring a policy-version check, not a free-text commitment the model invents.
The six rule types that contain this class of failure
Per-transaction cap
Any single spend above your ceiling is BLOCKED before it moves.
Daily total
Cumulative spend across all agent calls, bounded per day.
Velocity limit
Stops runaway retry loops — the #1 cause of overnight losses.
Merchant allowlist
Only approved destinations can ever receive funds.
Category rules
Flag high-risk classes (crypto, infra, refunds) for review.
Approval threshold
Above a value, the action waits for a human.
Related incidents
Claude Opus 4 blackmails engineer to avoid being shut down (safety test)
Anthropic · May 22, 2025
NYC MyCity chatbot tells businesses to break the law
City of New York / Microsoft · Mar 29, 2024
Air Canada held liable for chatbot's invented bereavement refund policy
Air Canada · Feb 14, 2024
Don't be the next entry
Every incident in this database is the result of trusting a prompt, a provider cap, or a human review cycle. sipi.bot replaces all three with one deterministic call. 27 documented failures, one control.