Token Cache Hit Savings
Providers discount repeated input tokens — often by an order of magnitude. For agents with stable system prompts, caching is the cheapest optimization on the bill.
How cache pricing works
Repeated context blocks get discounted input rates — cache hits cost a fraction of fresh input.
The discount applies to the cached prefix, not the whole call.
What agents benefit
Agents with stable system prompts and tool definitions hit the cache on every turn.
Workflows that re-send the same document or conversation benefit most.
The honest range
Cache-hit savings vary by provider and pattern — from modest for short prompts to order-of-magnitude for large stable contexts.
Check your provider's current cache pricing; the shape (cached prefix × discount) is universal.
At a glance
| Pattern | Cache benefit | Agent action |
|---|---|---|
| Stable system prompt | High | Keep prompts static |
| Repeated documents | High | Cache before re-sending |
| Fresh long contexts | None | Cap context size |
| Short random calls | Low | Not the lever |
FAQ
Do cache discounts apply automatically?
Mostly — providers cache automatically with same-prefix reuse. Some offer explicit caching APIs.
What's the typical saving?
It varies by provider and prefix size; the honest answer is 'sometimes an order of magnitude, sometimes nothing' — measure your own pattern.
Does caching replace spend control?
No — caching reduces cost per call; the firewall controls whether the call happens at all.
Related
Stop the next $12,400 night.
One API call (or MCP tool) in front of every agent transaction — APPROVED, BLOCKED, or FLAGGED, deterministic, ~5 ms, fully logged.
See plans — from $99/mo Try a live check