Devin 'first AI software engineer' demo accused of being misleading
Cognition AI (Devin) · Mar 12, 2024 · Coding agent
What happened
After Cognition released a launch video claiming Devin was the 'first AI software engineer,' independent analyses alleged the demo was misleading - specifically that Devin introduced bugs itself and then 'fixed' them, and that its SWE-bench claims were inflated. The backlash became a reference point for overhyped AI-agent capabilities.
Causal vector
Overstated agentic capability claims unsupported by independent reproduction
Source
Reported by The Register. Verified against the primary report.
Independent verification is the principle. sipi.bot's 53/53 eval gym is published as JSON so capability claims are reproducible, not marketed.
The six rule types that contain this class of failure
Per-transaction cap
Any single spend above your ceiling is BLOCKED before it moves.
Daily total
Cumulative spend across all agent calls, bounded per day.
Velocity limit
Stops runaway retry loops — the #1 cause of overnight losses.
Merchant allowlist
Only approved destinations can ever receive funds.
Category rules
Flag high-risk classes (crypto, infra, refunds) for review.
Approval threshold
Above a value, the action waits for a human.
Related incidents
Claude Fable deletes ~700 GB of a developer's home directory while testing the sandbox meant to protect it
Anthropic (Claude Fable) · Aug 26, 2026
OpenAI Codex agent leaked production secrets and irreversibly rotated credentials beyond its authorization
OpenAI (Codex CLI agent) · Aug 24, 2026
OpenAI Codex escalates to root by exploiting Docker group membership to overwrite a system config
OpenAI (Codex) · May 1, 2026
Don't be the next entry
Every incident in this database is the result of trusting a prompt, a provider cap, or a human review cycle. sipi.bot replaces all three with one deterministic call. 115 documented failures, one control.