A global survey of 1,500 IT and DevOps/platform engineering decision-makers published today finds 85% planning to increase their investments specifically to support artificial intelligence (AI), with well over a third (37%) running AI applications on their current IT infrastructure will be a significant challenge.
Conducted by the market research firm Vanson Bourne on behalf of Nutanix, the survey also finds 80% of respondents expect hybrid IT environments to be the most beneficial to their ability to manage applications and data.
That’s critical because while most AI models will be trained in the cloud, the bulk of the inference engines running those AI models will run in some type of on-premises IT environment, says Lee Caswell, senior vice president for product and solutions marketing at Nutanix.
In addition, organizations are using techniques such as retrieval-augmented generation (RAG) require organizations to be able to securely expose data that often resides in an on-premises IT environment to a large language model (LLM) that drives a generative AI application, he adds. “All that data resides in separate repositories, says Caswell.
A full 90% of respondents also said they are taking a “cloud smart” approach to their infrastructure strategy by leveraging the environment best suited for each of their applications. Nearly all (95%) have moved applications from one environment to another over the past year, with security and innovation cited as the top drivers for moving applications.
Nearly half (46%) said their organization has already implemented a hybrid cloud or hybrid multicloud IT environment, the survey finds. A full 90% said strengthening their edge strategy will be an important priority for their organization in 2024, with nearly three quarters (72%) reporting their organization plans to increase investments in edge computing throughout 2024.
Flexibility (41%) is cited as primary reason for opting to deploy applications in the cloud or an on-premises IT environment. However, more than half (51%) noted their hybrid multicloud environments are not fully interoperable.
Finally, 88% noted sustainability is an issue for their organization as they add workload to their IT environments.
It’s not clear to what degree organizations are managing the training of AI models separately from the deployment of inference engines needed to run them, but much of the latter efforts are increasingly being managed by IT teams running, for example, Kubernetes clusters. The more organizations invest in AI, the more likely it becomes they will deploy additional Kubernetes clusters.
Longer term, there may even come a day when current platform classifications are rendered obsolete as IT teams adopt a more federated approach to managing data.
In the meantime, IT teams would be well-advised to start estimating the number of AI models their organization plans to train and deploy. Training an AI model only to discover the organization needs a lot more compute capacity to run them is going to have a major impact on any return on investment (ROI) goals that might have been established.
The challenge now is to get ahead of those infrastructure requirements before, later on, insufficient IT resources wind up being a source of unneeded contention.
