LLM Cost Optimization Playbook
Seven moves, in order of impact. Each one is measurable and most are free.
The moves
1. Right-size the model — use frontier models for hard tasks, compact models for the rest.
2. Cache stable prefixes — system prompts and tool definitions hit cache discounts.
3. Trim context — summarize history, cap tool outputs, drop dead weight.
4. Cap retries — velocity limits stop loops from multiplying cost.
5. Set per-agent budgets — every agent gets a ceiling.
6. Watch overage tiers — set caps below included allotments.
7. Measure — the audit log shows what actually costs.
Why this order
Model choice and caching change the per-token economics; the rest stop the volume from escaping.
The two rules that do the most
per_transaction_cap max=200 action=block
velocity_limit max=10/60s action=blockStop the two most common runaway patterns in one config.
FAQ
Which move saves the most?
Model right-sizing, usually — frontier models cost multiples of compact ones for the same task.
How do I know what's costing me?
The audit log records every decision with amount and merchant; provider dashboards show token spend.
Do I need all seven?
No — start with caps and velocity (two rules), then optimize model choice and caching.
Related
Stop the next $12,400 night.
One API call (or MCP tool) in front of every agent transaction — APPROVED, BLOCKED, or FLAGGED, deterministic, ~5 ms, fully logged.
See plans — from $99/mo Try a live check