TL;DR — Key Takeaways
- AI costs are forcing changes: 89% of respondents have limited or redesigned AI use cases because of rising token consumption.
- Governance remains a scaling barrier: 75% say governance and compliance constrain AI expansion, while 84% say residency, sovereignty or jurisdiction-specific requirements influence workload placement.
- Financial returns remain uneven: Only 23% say AI generates more value than it costs; 58% report roughly equal value and costs, and 19% say costs exceed value.
A global survey of 1,500 IT leaders and decision-makers published today finds that 89% of respondents have limited or redesigned use cases involving artificial intelligence (AI) because of cost issues stemming from rising token consumption.
Conducted by Wakefield Research on behalf of Cloudera, the survey also finds that every respondent reports their organization has experienced higher or more volatile AI costs, with 99% moving to reduce them by, for example, moving a workload to another IT environment (45%) or reducing processing frequency, model size, context size, or output volume (40%). Just under a quarter (24%) have paused or canceled an AI use case altogether.
Additionally, nearly all respondents (98%) said their organization has pulled back on AI deployments due to failure, hallucinations, or unexpected behavior, with 68% of respondents needing to pull back workloads sometimes or frequently.
The survey also finds that data residency, sovereignty, or jurisdiction-specific requirements are influencing where AI workloads are deployed for a full 84% of respondents. More than half of respondents (58%) also noted that moving workloads requires moderate or major redesigns.
Three quarters of respondents (75%) also noted that governance and compliance issues are limiting their ability to scale AI deployments, with 55% reporting that governance costs are increasing faster than expected.
Overall, a total of 58% said the value their AI initiatives generate is approximately equal to what they cost, while just 23% say AI currently generates more value than it costs. Another 19% said costs outweigh the value generated. More than a quarter (27%) report little or no measurable impact from their AI initiatives, while 41% have encountered higher costs than expected.
A full 90% said they can measure end-to-end AI costs at least somewhat accurately, but only 27% say they can do so with a high degree of accuracy, the survey finds. Only 19% continuously monitor their costs.
Collectively, these issues suggest that the average enterprise is still wrestling with many AI fundamentals, says Jeff Healey, senior vice president of product marketing for Cloudera. Those issues include everything from optimizing costs and improving scalability to addressing latency, security, compliance and governance requirements, he adds. For example, not every workload requires access to the latest frontier AI model when an open weight model deployed in an on-premises IT environment might prove to be much more cost-effective, he adds. Managing token costs has become an arbitrage exercise, notes Healey.
In other cases, organizations need to determine if using AI makes sense in the first place, he adds. “You don’t need to use AI for everything,” says Healey.
The degree to which each of these issues is a challenge naturally varies. In fact, more than three-quarters (76%) say it is easy to apply consistent data governance, security and access policies across their AI environments. Just over half (54%) believe their organization has struck an effective balance between the speed of AI adoption and appropriate governance.
Nevertheless, the survey finds that nearly half of respondents (49%) would strengthen governance from the beginning, while 46% would have invested in data quality earlier. A similar percentage (45%) would have invested in both modernizing infrastructure and employee training sooner.
In general, survey respondents cite resilience (42%), sovereignty and residency requirements (38%), cost reduction (38%), and proximity to data (37%) as the top reasons for running AI workloads in multiple IT environments.
Despite all these issues and concerns, however, investments in AI continue to be made. Nearly two-thirds (64%) of respondents say their organizations spend at least $500,000 annually on AI initiatives, while 28% spend $1 million or more. In fact, 43% report greater-than-expected productivity.
Nearly two thirds (63%) also said AI is an important input in organizational decision-making, with just over a quarter (27%) noting that AI regularly shapes important decisions or serves as a primary driver. More than three quarters (78%) require human approval for at least some AI-driven decisions, but the circumstances vary. More than half (52%) trigger approval when a confidence, risk or value threshold is crossed, while 34% require it for legally regulated or designated high-impact decisions and 30% for decisions affecting customers, employees or regulated outcomes.
It’s apparent that while there are still plenty of AI challenges ahead the benefits still outweigh the sum total of the costs that organizations are moving to rapidly contain.
