TL;DR — Key Takeaways
- Harness adds AI spending controls: AI Budgets enables organizations to enforce spending policies for individual users, teams, developers, models and projects.
- Token visibility supports cost management: Insights into input, output and reasoning token usage help organizations identify costly tasks and improve prompt efficiency.
- Harness reports internal savings: The company says it reduced AI coding costs by 37% using the capability internally.
Harness this week added an ability to implement spending policies using an artificial intelligence (AI) agent it previously developed to track how much infrastructure resources are being consumed.
The AI Budgets capability that has been added to the Harness Cost Management Agent makes it possible to limit how much resources are allocated to a specific user, team, developer, model, or project. Additionally, organizations are provided insights into the cost of specific tasks, such as the number of tokens used for input, output and reasoning to provide a deeper dive into the elements of workflow that are generating the most cost.
Organizations can set up different levels of spending policies that are automatically enforced on a weekly or monthly basis. Alternatively, they can apply policies to limit access to more expensive AI model services.
In effect, AI Budget makes it possible to apply many of the same FinOps practices that many have already applied to the consumption of cloud infrastructure resources.
Developed initially as a tool that Harness uses internally to keep a lid on AI coding costs, the AI Budget capability makes it possible to more proactively reduce costs by applying policies to specific end users, says Patrick Brogan, director of the FinOps advisory service for Harness. “We were able to generate a 37% savings,” he adds.
Whenever the number of tokens consumed exceeds limits, an end user is then presented with a button through which they can request additional allocations, notes Brogan.
AI Budgets require about two days to learn an environment before alerts are generated. Once those alerts start to be shared, most end users leverage the insights provided to create more efficient prompts, noted Brogan. In the absence of those insights, end users will often consume tokens as if they were a virtually unlimited resource until they run out.
Many organizations, after initially encouraging end users to experiment with AI as much as possible, are now trying to rein in AI costs that have in many cases quickly ballooned. Accenture, for example, has introduced a funding policy that restricts employee access to Anthropic’s Claude AI tools, capping central funding at $29 per worker per month to curb costs.
It’s not clear how many other organizations are following suit, but it’s apparent there is now much greater sensitivity to costs. The challenge is that adoption of AI tools and models tends to be highly decentralized, which makes it challenging to track how much a specific department or business unit might be spending versus another. The one thing that is certain is not all prompts are of equal value to the business so some care needs to be taken to ensure that projects that are crucial to the business are not being starved for tokens that are being wasted on more trivial tasks.
In effect, many organizations are now essentially rationing tokens. The challenge, of course, is making sure that whatever tokens are consumed drive a meaningful return on investment (ROI) for the business.
