Freysa AI agent talked into transferring its entire $47,000 prize pool
Freysa (adversarial agent game) · Nov 29, 2024 · General agent
What happened
Freysa, an autonomous on-chain AI agent, was given one hard instruction: never transfer its prize pool. Participants paid per message to try to talk it into doing exactly that, growing the pool to roughly $47,000 across 195 participants. One user then crafted a message that convinced the agent to call its transfer function anyway, and Freysa sent the entire prize pool in ETH to the challenger.
Causal vector
Persuasion attack on a system prompt guarding a live transfer function — the prompt was the only control
Source
Reported by OECD.AI Incidents Monitor. Verified against the primary report.
Freysa's only defence was its system prompt — precisely the layer that failed. A merchant allowlist enforced outside the model, with zero approved destinations, returns BLOCKED on the transfer call no matter how persuasive the message is. The policy, not the prompt, has to hold the money.
The six rule types that contain this class of failure
Per-transaction cap
Any single spend above your ceiling is BLOCKED before it moves.
Daily total
Cumulative spend across all agent calls, bounded per day.
Velocity limit
Stops runaway retry loops — the #1 cause of overnight losses.
Merchant allowlist
Only approved destinations can ever receive funds.
Category rules
Flag high-risk classes (crypto, infra, refunds) for review.
Approval threshold
Above a value, the action waits for a human.
Related incidents
NeuralTrust: 34.9% of enterprises report confirmed AI agent security incidents
NeuralTrust · Jun 1, 2026
HUMAN Security: AI agent traffic grew 7,851%, 8x faster than human traffic
HUMAN Security · Jan 1, 2026
Don't be the next entry
Every incident in this database is the result of trusting a prompt, a provider cap, or a human review cycle. sipi.bot replaces all three with one deterministic call. 67 documented failures, one control.