Companies are bullish on AI, but cost is a thorny issue keeping many from participating in the play. A lot of organizations feel impeded by AI’s heavy demand for resource and model deployment constraints that lead to tricky situations like vendor lock-ins and cost blowouts.
A McKinsey report estimates that the cost of a single training run ranges somewhere between $4 million and $200 million. Computation costs account for a significant portion of that.
Clarifai recently released a new compute orchestration capability that allows enterprises to efficiently manage the infrastructure and optimize their compute spend for AI.
Announced for public review in December 2024, the new feature lets users orchestrate AI workloads hosted in any infrastructure – cloud, on-prem, VPC, SaaS, or air-gapped environments – using an unified control plane.
It works seamlessly on any hardware and with any AI model, giving companies an open playing field compared to the closed and constrained single-vendor ecosystems.
Founded in 2013, Clarifai spent over a decade optimizing compute stacks, and supporting public sector and government companies with full-stack tools for their AI.
In a briefing with Tech Field Day, a business unit of The Futurum Group, Patrick Lin, product manager for Clarifai, told that the feature originally started as an internal project for improving Clarifai’s own inference system. Once it produced impressive results, the next step was to make it available to customers to help optimize their resource and compute costs.
Cost optimization with Compute Orchestration is delivered chiefly through automated resource management, a function it offers through a series of features like model packing, customizable auto-scaling, and dependency management.
To start, customers need to select a model and the compute of their choice. Clarifai takes care of all orchestration tasks behind the scenes.
The control plane allows users to regulate access to AI resources, monitor performance and optimize costs at their end. Simultaneously, Clarifai’s automated optimization and tuning kicks off, keeping usage at the minimum level.
According to Clarifai, the optimizations can lead to a 3.7x reduction in compute usage, which translates to a minimum of 60% cost reduction, which can go as high as up to 90% in select cases.
Besides delivering cost-efficiency, Compute Orchestration also promises optimized AI performance through dynamic resource allocation.
Lin told that Clarifai can support over a million input per second.
A gallery of pre-built, pre-trained models developed in-house gives customers the option to use models that work out-of-the-box as well as be trained for specific use cases.
Clarifai does model serving with many popular open models, as well as works with uploaded customer models, said Lin.
More than 2.5 million companies actively use or are dabbling in AI, worldwide. The global market size which stood at $127B in 2024 is expected to grow to over $143B by the end of 2025.
