TL;DR — Key Takeaways
- Kong expanded its AI management capabilities with cost controls, observability and enhanced management of model and MCP interactions.
- IT teams can establish budgets, track usage, allocate costs and identify savings through caching, compression and intelligent model routing.
- Kong Token Vault is generally available, allowing AI agents to access authorized resources without directly holding downstream credentials.
Additionally, Kong has expanded the reach of its Kong AI gateway to AI models and Model Context Protocol (MCP) servers, in addition to adding an ability to authorize access via the Kong Identity platform.
Finally, Kong has made Kong Token Vault, an addition to its portfolio that brokers authorized downstream credentials at runtime rather than requiring agents to hold them directly, generally available.
Reza Shafii, senior vice president of product for Kong, said that as more organizations focus on limiting the cost of AI, otherwise known as tokenization, there is clearly a greater need for intelligent routing that directs prompts to the AI model best suited to respond at the lowest cost possible. In fact, many organizations are already adding open-weight models to reduce their dependency on frontier models that are significantly more expensive to invoke, he added.
Those organizations can now use the Kong application connectivity platform to establish budgets, track usage, and enable showback or chargeback to allocate costs by input and output tokens, cached tokens, and service tiers. IT teams can also identify opportunities to further reduce cost by applying optimization techniques such as caching, compression and reasoning controls.
The observability capabilities, meanwhile, provide visibility across complete AI sessions and interactions, allowing teams to follow activity across models, tools, agents, MCP interactions and policies in a way that makes it possible to investigate behavior, latency, errors and cost.
IT and platform engineering teams are at the forefront of efforts to both rein in costs and apply the governance policies needed to ensure AI agents do not engage in any rogue behavior, said Shafii.
The challenge, of course, is that the pace of AI adoption has, thus far, exceeded the ability of many IT teams to effectively manage it. However, as the “token mania” era gives way, there is now a lot more focus on governance and cost, said Shafii. “IT people are not going away,” he says. “They will be a big part of this.”
One way or another, just about every organization is revisiting their IT strategy with an eye toward striking a balance between potentially deploying thousands of AI agents and the inherent risks those AI agents create. Given the fact that those AI agents typically perform tasks at machine speed, the only way to effectively apply governance policies is to route that traffic through some type of centralized gateway.
Hopefully, the level of supervision can be applied before there is a major incident. In the meantime, IT teams should assume that many more AI agents than they can currently observe have already been deployed. The issue then becomes first discovering those AI agents and preventing them from accessing external resources unless specific permissions have been granted.
Like it or not, it’s only a matter of time before auditors begin to force a larger AI governance conversation. The paradox, of course, is they will undoubtedly leverage AI to discover the activity of other AI agents that may be violating one or more compliance mandates that could lead to fines being levied that far exceed any penalties that have gone before.
