TL;DR — Key Takeaways
1. AI infrastructure’s biggest challenge isn’t traffic volume — it’s interaction cost. A single user prompt can trigger dozens of model calls, API requests, retrieval operations and security checks, each consuming resources and adding latency.
2. Enterprise AI spending is shifting toward inference and RAG. F5 reports that approximately 65% of its current AI Data Delivery opportunities focus on inference and RAG deployments, compared with a training-heavy market last year.
3. Intelligent application delivery is becoming essential. Enterprises need smarter model routing, caching, token governance, API protection and efficient data access to reduce costs and deliver AI responses without unnecessarily expanding infrastructure.
If you’ve spent any time reading about AI infrastructure lately, you’ve probably heard that AI is going to create an explosion of network traffic.
That’s true.
But I don’t think traffic is the real story. The biggest challenge AI introduces isn’t moving more data. It’s that every AI interaction is becoming more expensive.
A few years ago, a user request probably meant an application queried a database and returned an answer.
Today, that same request can trigger an entire chain of interactions across models, enterprise applications, data stores, APIs, and security controls before anything comes back to the user.
The user submits one prompt. The infrastructure executes dozens of conversations. That’s the architectural shift we’re beginning to see.
We’re already seeing this shift in our own AI Data Delivery business. Last year, roughly 92% of our AI Data Delivery opportunities centered on training workloads. Today, approximately 65% are focused on inference and RAG deployments. Customers are asking less about building GPU clusters and more about operating AI applications efficiently once they’re in production.
That transition matters because inference doesn’t just increase AI activity. It fundamentally changes how applications interact with infrastructure.
The challenge isn’t simply more traffic. It’s the growing number of interactions behind every request. Every model invocation consumes compute and tokens. Every API call adds latency. Every retrieval from a vector database or object store consumes resources. And every millisecond a GPU waits for data is another millisecond you’re paying for hardware that isn’t doing useful work.
I think we’re asking the wrong question. We’ve spent years measuring infrastructure in terms of throughput, bandwidth, and utilization.
AI introduces a new metric.
Efficiency.
The question isn’t “Can my infrastructure move more traffic?”
It’s “How efficiently can my infrastructure deliver intelligence?”
Consider what’s happening inside a typical enterprise AI workflow. A single prompt might trigger a planner agent, call an LLM, retrieve context through RAG, query multiple enterprise applications, apply governance policies, perform security checks, and generate a response.
Each step creates another decision:
- Which model should handle this request?
- Does this request even need a model or can it be answered from cache retrieval?
- Is this workflow within the token budget?
None of those questions are solved by adding bandwidth. They’re solved by making smarter decisions throughout the application path. We’re already seeing evidence that this shift is underway.
According to Deloitte, nearly two-thirds of large enterprises expect to consume more than 10 billion AI tokens every month within the next few years. As recently as February, AT&T was forced to rethink its AI orchestration when its average daily token usage reached 8 billion per day. These are so much more than adoption statistics. This is an economic step change. AI doesn’t scale like traditional web applications. Every additional interaction has a measurable infrastructure cost, because every token represents compute and every inference request has a price.
That changes the role of the application delivery layer.
Historically, application delivery focused on availability, performance, and security. Those responsibilities aren’t going away, but they’re expanding. The application path is becoming the place where organizations optimize AI interactions by routing requests, enforcing policy, protecting APIs, governing token consumption, and improving efficiency before a request ever reaches the model.
That’s why I think the conversation around AI infrastructure is about to change. The winners won’t necessarily be the organizations with the largest GPU clusters or the fastest networks. They’ll be the organizations that can deliver intelligence most efficiently.
Because in the AI era, the challenge isn’t moving more traffic. It’s making every interaction count.
For organizations beginning this journey, the next step isn’t simply adding more infrastructure. It’s designing an application architecture that can intelligently route, secure, govern, and optimize every AI interaction. Our AI Reference Architecture explores what that looks like in practice, providing a blueprint for building AI applications that are secure, scalable, and efficient from the start.
