Surging inference demands mean enterprises are flocking to AI-optimized infrastructure to keep pace
New figures from Gartner point to a surge in AI-optimized infrastructure spending over the next year. A senior analyst at the firm told ITPro it underlines how it is driving a “fundamental shift” in cloud consumption and design.
From our partners:
Recent analysis from the firm found spending on this front is projected to grow 96% across 2026, reaching a total of $42 billion as enterprises look to overhaul infrastructure with the specific aim of dealing with cumbersome AI workloads.
Hardeep Singh, senior principal research analyst at Gartner, told ITPro the consultancy expects this trend to continue as adoption rates rise. Indeed, Gartner projects the market to “sustain high growth” and reach $66 billion in spending.
“There is a fundamental shift in how cloud infrastructure is being designed and consumed,” Singh told ITPro. “We are transforming from traditional general purpose cloud computing towards purpose-built AI infrastructure designed specifically for large language models (LLMs) and agentic workloads”.
A key factor behind the optimized infrastructure push lies in the changing shape of AI use, Singh noted. The technology is maturing and operationalization of AI, particularly agentic AI, is “accelerating inference demand” across the board.
Agentic AI has a far larger token consumption rate compared to traditional chatbot-based AI interactions, which in turn increases inference demand.
Research from Signal65, for example, found agentic workloads consume “orders of magnitude more tokens”, with consumption rates anywhere between four to 15-times more.
“You see [with] agentic AI it has a variable consumption,” Singh told ITPro. “It would be one prompt triggering autonomous model calls, and that would lead to a huge amount of compute consumption.”
Gartner projects that inference demands will surge over the next year on the back of increased agentic AI adoption. Indeed, the consultancy noted that spending on this front will reach $23.3 billion, surpassing the $19 billion spent on AI inference.
The fact that inference spending will exceed training for the first time is noteworthy, Singh told ITPro. Many enterprises are still in the early stages of agentic AI adoption though. As projects mature, this is certainly expected to drive inference demands even further.
Gartner ranks AI use based on a level system, with basic chatbots at the first level, and fully autonomous at level five, the highest rating. Most businesses aren’t even close to that level, he said.
“It will still take some time to go and reach the autonomous side,” Singh told ITPro.
“We are still in that L1, L2, L3 kind of scenario where experimentations are happening, trying to understand which use cases fit in which scenario, and then how the agents [are] going to solve the complex business problems.”
Given the current state of agentic adoption and projected advances on this front, AI-optimized infrastructure will be even critical for meeting demand.
Gartner figures show that 55% of spending on this front will directly support inference. This is set to reach 59% by 2027.
Changing priorities
According to Gartner, the growing volume of inference workloads will ultimately “reshape cloud investment priorities”. Singh noted that this can already be observed in cloud strategies across a range of industries.
Research from IDC in late 2025 predicted a sharpened focus on hybrid and even private cloud models due to a combination of factors, most notably “more predictable cost models”.
Singh told ITPro he expects this focus to continue as agentic adoption accelerates, particularly with the rise of multi-agent frameworks in which they rely on several separate underlying AI models.
“The important thing will be how enterprises want to use multi-agents backed by multiple models, so multimodality and multi-models,” he said.
“Both these things are important in the context where some of the data workloads [are] moving to the cloud, training the large language model, then deploying on the edge on prem to get the inference and real-time monitoring.”
“Hybrid and multi-cloud will be more picked up in these scenarios, because some enterprises know that it is difficult to train the model on-prem because of the amount of capacity it needs.”
FOLLOW US ON SOCIAL MEDIA
For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!
Our humans need coffee too! Your support is highly appreciated, thank you!
