How to Reduce AI API Costs
There are only two ways to reduce AI API costs: pay less per call and make fewer calls. Most teams focus on the first and ignore the second. The second is where the biggest savings are.
Lever 1 — Pay less per call
Right-size your models
The single biggest per-call cost reduction available. Most teams run everything on one model (usually GPT-4o or Claude Sonnet) when most of their work could run on a budget-tier model at 16–60× lower cost.
| Move from | Move to | Savings | When |
|---|---|---|---|
| GPT-4o ($2.50/M) | GPT-4o-mini ($0.15/M) | 16× | Classification, routing, simple extraction |
| Claude Sonnet ($3/M) | Claude Haiku ($0.25/M) | 12× | Summarization, formatting, basic Q&A |
| Claude Opus ($15/M) | Claude Sonnet ($3/M) | 5× | Most coding and research tasks |
| GPT-4 ($10/M) | GPT-4o ($2.50/M) | 4× | Most tasks that don't need GPT-4's specific strengths |
Full pricing in the token cost benchmark.
Cache repeated prompts
If your agent sends the same (or similar) prompts repeatedly — system prompts, few-shot examples, common queries — cache them. Most LLM gateways (LiteLLM, Portkey) support prompt caching. Anthropic and Google also offer native prompt caching for repeated context.
Compress your context
Long system prompts and large context windows are a hidden tax. A 4,000-token system prompt on GPT-4o costs $0.01 per call before the agent does anything. Trim it. Move boilerplate to cached context. Use prompt compression tools for long documents.
Lever 2 — Make fewer calls (the bigger savings)
Kill retry loops with a velocity rule
A velocity rule caps the number of transactions per time window. When the agent hits the cap, further calls are blocked. The retry loop dies on the Nth attempt instead of running for 90 seconds.
{"rule_type": "velocity", "params": {"max_count": 10, "window_minutes": 60}, "action": "BLOCKED"}This is the single highest-ROI cost-reduction lever. One rule prevents 44% of runaway incidents.
Block unnecessary spend with a merchant allowlist
If your agent is spending at vendors you didn't intend to approve — novel vendors it discovered, prompt-injection-driven routing — a merchant allowlist stops that spend entirely.
{"rule_type": "merchant_allow", "params": {"allowed": ["openai.com", "anthropic.com"]}, "action": "BLOCKED"}Cap categories independently
Don't let one category eat the budget. If your agent is supposed to spend on LLM calls but is also buying compute or ads, cap those categories separately.
{"rule_type": "category_limit", "params": {"category": "ads", "max_amount": 50}, "action": "BLOCKED"}The combined savings
A typical team that implements both levers sees:
- 40–60% reduction from right-sizing models (move routine work to budget tier).
- 20–30% additional reduction from killing retry loops with a velocity rule.
- 5–10% additional reduction from blocking unnecessary spend with allowlists and category caps.
- Total: 65–90% cost reduction without changing what the agent does, only how it spends.
The tools
| Lever | Tool |
|---|---|
| Right-size models (routing) | LiteLLM, Portkey, OpenRouter |
| Cache prompts | LLM gateway (LiteLLM, Portkey) |
| Observe spend | Helicone, Langfuse |
| Kill retry loops + enforce budget | sipi.bot (pre-spend firewall) |
Most production stacks run all four. They are complementary layers, not alternatives.