New CLI, SDK and APIs give AI and platform teams one place to manage token spend on approved models, falling back to private models automatically, while respecting DevX and keeping agents working.
SAN FRANCISCO, July 15, 2026 /PRNewswire/ — Soaring token costs are pushing engineering teams to manage spend, and Tetrate is rolling out a new capability to solve the problem as part of its Agent Router Enterprise, the AI gateway proven at production scale. The feature accesses a token-brokering control plane via CLI, SDK and set of APIs to enable distributed inference, which defines the mix of frontier, private and edge models a company might run across many regions and providers. Now, Agent Router Enterprise ties every inference request to approved policies for spend (as well as availability and sovereignty) before it routes.
A token broker is a service on top of the AI gateway that sits between developers and agents, and the models they inference on. The broker evaluates each developer or agent request against a set of business, technical and governance rules (how much budget is left, which models are approved, which region the work must stay in) and sends each request to the model best optimized to the rules.
Agents spend faster than teams can cap
Agents have changed how enterprises consume inference. A single request can fan out into dozens of model calls before it returns an answer, and the model deciding how many to make has little sense of what they cost or how well those requests adhere to governance policy.
The pressure is widely documented. In the FinOps Foundation’s 2026 State of FinOps survey, 98 percent of practitioners now manage AI spend, and “FinOps for AI” ranks as their top forward-looking priority. The paradox is that unit prices keep falling even as bills keep climbing: Stanford’s 2025 AI Index found the cost of GPT-3.5-level inference dropped more than 280-fold in two years, yet total spend rises because agents multiply the number of calls. Each token gets cheaper, while the invoice gets bigger.
To manage this, most teams reach for the open-source proxy they started with, then blame their own configuration when it falls short. The limit is architectural. That proxy was built as a local gateway in front of a handful of models, not a control plane across a distributed fleet, so it cannot discover models across regions or hold a single budget over all of them. What remains is manual gatekeeping, one model and one region at a time, with the AI or platform team as the bottleneck between every developer and every endpoint.
Token brokering from one control plane
Agent Router Enterprise now closes that gap because it is an AI gateway built for distributed inference. Its APIs allow AI teams to register and govern every model it offers, frontier and private, and set budget policy, approved regions and fallback order. Developers request the nearest approved model through the SDK, and Agent Router Enterprise enforces that policy on every token in between, and the team serves any approved model without standing in as the gatekeeper.
Budget control is the core of the offering, because spend is what the engineering team answers for when costs run over. Agent Router Enterprise measures token spend against policy and trips a circuit breaker the moment a budget is crossed, then falls back to an approved alternative. That fallback is often a private model that costs less than a frontier one and keeps inference inside the company’s own environment. That holds spend down while keeping the request in an approved region and on a healthy endpoint without manual intervention.
A new feature in Tetrate Agent Router Enterprise is a distributed inference control plane that sits between developers and agents, and the models they consume. The administrator sets policy in the management plane, and Agent Router Enterprise enforces it on every request to manage soaring token costs.
Designed to drop into workflows teams already run
The feature assumes two kinds of users and keeps them out of each other’s way. The AI team that owns budget governance sets policy once (approved models and regions, budgets, fallback order) through the APIs. Developers and agents keep requesting the nearest approved model through the SDK without the burden of implementing their own controls. When a budget trips, the feature redirects to an approved alternative rather than returning an error or forcing anyone to rewrite code. For organizations that have already committed to building with AI and are absorbing the back-and-forth of cost pressure, that is the point: a control plane that slides into existing pipelines.
That framing tracks how analysts now describe the market. McKinsey’s 2026 analysis of sovereign AI estimates that 30 to 40 percent of AI spending could be shaped by sovereignty requirements, valuing the market at $500-600 billion by 2030. It also finds the demand is selective, concentrated in workloads with sensitive data or regulatory exposure, and that providers succeed when they make sovereignty “concrete and easy for enterprises to adopt at the workload level.” Agent Router Enterprise is built for exactly that: sovereignty is one policy lever applied per workload, alongside cost and availability, not an all-or-nothing rebuild.
Proven at scale on Envoy AI Gateway
The new capability holds up at scale because it runs on the open source Envoy AI Gateway project, which Tetrate co-created and serves as the chief upstream maintainer. Tetrate Agent Router Enterprise enforces it across thousands of developers in hundreds of environments and regions.
“Agent token spend is the one line item the engineering organization can’t easily cap for CFOs, even though the entire business is looking to them for answers,” says David Wang, head of product at Tetrate. “With this update, admins set a budget once, and it’s enforced on every token, falling back to an approved model whenever spend or availability breaks, so cost never gets ahead of policy, agents can never incur unexpected runaway costs, and developers keep working the way they already do.”
“At Sunny Benefits, managing inference cost across a range of models is something our platform team owns and answers for,” says Sashi Desikan, CTO at Sunny Benefits. “Agent Router Enterprise lets us set a budget once and have it enforced automatically, with fallback to an approved model when we need it. Our developers keep getting the nearest approved model without us reviewing every request.”
Availability
The new feature is now generally available as part of Tetrate Agent Router Enterprise, at no additional charge to subscribers. It is not sold or priced as a separate module.
About Tetrate
Tetrate builds Tetrate Agent Router Enterprise, the AI gateway proven at production scale for distributed inference, giving enterprises one place to govern token spend, model access and sovereignty across models, regions and providers. A primary upstream contributor to the Envoy project and co-creator of Envoy AI Gateway, Tetrate brings that foundation to the AI infrastructure enterprises run in production.
MEDIA CONTACT:
Cristin Connelly
Cathey.co for Tetrate
cristin@cathey.co
View original content to download multimedia:https://www.prnewswire.com/news-releases/tetrate-adds-token-brokering-capability-for-ai-code-gen-cost-management-through-agent-router-enterprise-302826151.html
SOURCE Tetrate
Technology5 days ago
Coin Market5 days ago
Coin Market4 days ago
Technology5 days ago
Technology5 days ago
Coin Market4 days ago
Technology3 days ago
Technology1 day ago