Best LLM Gateways 2026
An LLM gateway routes, observes, and sometimes rate-limits model traffic. The best one for you depends on scale, stack, and whether you need a money gate too.
The gateways teams actually run
LiteLLM Proxy — the open-source standard for model routing and key management.
Kong AI Gateway — enterprise gateway with AI plugins.
Cloudflare AI Gateway — edge routing with analytics.
Portkey — gateway plus observability.
What gateways do
Route to models, manage keys, add caching and fallbacks.
Some rate-limit — but rate limits aren't dollar budgets.
Gateway vs spend firewall
A gateway manages how requests reach models.
A spend firewall decides whether money moves — for every merchant, not just model providers.
They compose: gateway for routing, firewall for the money.
At a glance
| Tool | Best at | Spend layer |
|---|---|---|
| LiteLLM Proxy | Model routing | External firewall |
| Kong AI Gateway | Enterprise gateway | External firewall |
| Cloudflare AI Gateway | Edge routing | External firewall |
| Portkey | Gateway + observability | External firewall |
FAQ
Is a gateway a firewall?
No — a gateway routes traffic. Rate limits stop requests, not purchases.
Do I need both?
At agent scale, yes — gateway for the model layer, firewall for the money layer.
Related
Stop the next $12,400 night.
One API call (or MCP tool) in front of every agent transaction — APPROVED, BLOCKED, or FLAGGED, deterministic, ~5 ms, fully logged.
See plans — from $99/mo Try a live check