NVIDIA this week donated OpenShell, an open source runtime for securely deploying artificial intelligence (AI) agents, to the Cloud Native Computing Foundation (CNCF).
Announced at the KubeCon + CloudNativeCon event, NVIDIA revealed it is also making contributions to Agent Substrate, an open source framework for operating long-running agent workloads at scale on Kubernetes clusters, which Google previously contributed to the CNCF.
Agent Substrate makes it possible to decouple stateful agent sessions and identities from active compute in a way that creates a snapshot of idle agents that is then stored and, when needed, restored in under 500 milliseconds to reduce costs by consuming infrastructure resources more efficiently. Giving every active agent its own Pod locks up memory and CPU resources while the agent sits idle. The end result is that the Kubernetes control plane needs to track millions of persistent objects it was not designed to manage.
NVIDIA is now committing to ensuring that Agent Substrate works with NVIDIA Dynamo for model inference and the KAI Scheduler for shared cluster resource allocation, both of which are core to its DSX AI Factory Platform.
Additionally, NVIDIA is making available NVIDIA Snapshot, an open source tool that restores checkpointed replicas in seconds without model reload or warmup. Design partners Crusoe and Saturn Cloud evaluated the tool on their workloads.
NVIDIA is also making available a public preview of an open source NVIDIA Kubernetes Engine (NVKE) and NVIDIA Kubernetes Recipes (NVKR) that are building blocks for delivering managed services. Mirantis, Netris, OpenNebula Systems, Rafay, Spectro Cloud, vCluster, Firmus and SK Telecom have also pledged support for this initiative.
Those contributions come on the heels of the contribution of gVisor, which isolates code inside a lightweight userspace application kernel that can be deployed as a container, and OpenRL, an open source project for training AI agents using reinforcement learning techniques, that Google made last week.
The overall goal is to increase the base of contributors to these projects as they are now advanced under the auspices of the CNCF, says Jonathan Bryce, executive director of the CNCF. Collectively, these projects address the constrained resources that IT teams are encountering in the AI era while at the same time addressing security and control issues, he adds.
It’s not clear how much support there is for these projects, but the overall goal is to make it easier to enforce policies as agents access IT infrastructure to access various tools. There is, of course, already no shortage of runtimes for securely deploying AI agents, but OpenShell now has support from hundreds of vendors.
Regardless of approach, the one thing that is certain is that AI agents are being deployed faster than IT and cybersecurity teams can manage. In fact, it’s only a matter of time before more of those teams encounter similar issues that surfaced following a series of security breaches involving rogue AI agents. The challenge now is putting in place the underlying infrastructure needed to secure AI agents before any similar incidents wreak additional havoc.
