The investment frenzy around generative AI (GenAI) is ticking up as a growing mass of global tech companies and promising startups rush to hard-launch products enhanced by the technology.
With the aim to help fast-track their ambitions, VMware by Broadcom, in collaboration with NVIDIA, has developed VMware Private AI Foundation with NVIDIA. The joint solution, a GenAI platform for the data centers, is designed to provide enterprises a low-effort and cost-optimized way to manage and deploy artificial intelligence (AI) workloads on private infrastructure, says the company.
Unveiled at VMware Explore 2023, and generally available since May of 2024, the solution is built on the latest VMware Cloud Foundation (VCF) v5.2.1. It includes deep learning VMs with virtual GPUs attached to them, infrastructure abstraction for an undemanding management experience, automation tooling stack for hands-off lifecycle management, and vector database for RAG workflows.
Adding to these is the NVIDIA AI Enterprise layer that offers capabilities to tune model performance and streamline model development and deployment.
Justin Murray, product marketing engineer, said that the goal is to provide data scientists an improved experience by offloading tasks like provisioning and infra management through automation, while supplying some of the tooling they like – both from NVIDIA, and those that are homegrown at VMware by Broadcom.
At the AI Field Day event last week, Murray delivered a technical presentation on VMware Private AI Foundation with NVIDIA where he gave a roundup of the new upgrades.
Since the launch of the product last year, there have been several new additions, but most of them came in the way of VCF 5.2.1. Murray highlighted model governance as an important update that was not available earlier. Model governance helps users determine the safety and suitability of new models by letting them test, secure and store large language models (LLMs).
“The safest place to do that in VMware Private AI is in a deep learning VM that’s in a safe zone that’s possibly disconnected from the Internet,” Murray said.
“Part of model governance is promoting that model from a simple straightforward deep learning VM into a more complex environment that is Kubernetes, a level where developers can now use it,” he added.
Also included is a vector database functionality that unlocks quick data query and real-time updates – a must for RAG workflows. Using VCF’s built-in automation stack, a vector database can be deployed and loaded with private business data easily, says VMware.
The self-service automation tooling, based on VCF automation, allow users to provision deep learning VMs for model testing, and Kubernetes clusters for app deployment.
“We want to remove the infrastructure from the data scientists as the concern,” said Murray. “Self-service automation is us doing all the work for [them] and setting up the Kubernetes cluster with all of the tooling so that they are ready to go in their Jupyter Notebook. Sometimes, we call this GPU as-a-service.”
VMware Private AI Foundation with NVIDIA also comes with a set of GPU monitoring capabilities that can be leveraged to maintain oversight on GPU utilization and the performance impacts of AI applications on the GPUs.
“If you can predict what workload, what model you’ll be doing in a month’s time, we can reserve that GPU for you ahead of time to guarantee that it’s there for you,” Murray said.
“Enterprises are sitting on a figurative goldmine that is a massive amount of data, and this data can be leveraged to extract insights to help make better business decisions; and better business decisions we hope would help produce better product and better outcome for services that you provide for internal and external customers,” said Yu Wang, technology product manager at the VCF division, at VMware Explore 2024, while talking about to the current GenAI boom.
GenAI uptake caused significant growth in the AI market last year, with the AI software market reaching $29.9B in Q2 of 2024, reported Futurum Intelligence. Bloomberg predicts that the GenAI market alone will overshoot a whopping $136B valuation by 2030.
Head over to Techfieldday.com to watch more presentations by VMware by Broadcom from the recent AI Field Day.
