Inference Economics: Gartner's 96 Percent

Hardeep Singh, a senior principal research analyst at Gartner, put a figure on a trend every cloud buyer already felt in their invoices: worldwide spending on AI-optimized infrastructure as a service will climb 96 percent in 2026, reaching $42.276 billion, up from $21.529 billion the prior year ¹. Inference alone accounts for $23.3 billion of that total, 55 percent of spend, and Gartner expects the split to tilt further toward inference in 2027, reaching 59 percent as training's early dominance fades ¹. Singh framed the driver plainly: "This growth is driven by continued demand for infrastructure to support large language model (LLM) training and rapid operationalization of AI across enterprise applications" ¹. Nvidia's own numbers corroborate the scale: data-center revenue hit $89.0 billion in the quarter ended late July 2026, up 117 percent year over year, chief executive Jensen Huang telling investors "compute is revenue" now, a line item generating income instead of a cost center still awaiting monetization ².

A paradox that resolves through volume

Spending nearly doubling looks incompatible with a second trend running in parallel: the price of a token has collapsed. A widely cited industry analysis puts the decline at roughly 95 percent over two years and close to 1,000-fold over three, tracing a pricing curve from GPT-4's $30 per million input tokens in March 2023 down to open-weight models charging around $0.10 by 2026 ³. Both trends are accurate simultaneously, and the reconciliation is volume: enterprises are running vastly more inference calls than the price collapse alone would predict, agent orchestration multiplying the number of model calls a single workflow now triggers. A customer-service agent handling a support ticket in 2023 made one model call; an agentic system handling the same ticket in 2026 might make a dozen, each one delegated to a sub-agent verifying a claim, checking a policy, or drafting a response for review. Gartner's dollar figure captures that multiplication; the per-token price chart captures only the unit economics underneath it.

Fireworks and Together race on model-serving speed

Lin Qiao built Fireworks AI on a specific bet: that serving open models faster and cheaper than the labs that trained them would command a durable market on its own. The bet paid out in scale terms this year — a Series D announced July 15, 2026 valued the company at $17.5 billion, alongside an announced $1 billion in annual recurring revenue ⁴. Qiao previously led PyTorch at Meta, and her founding team reads like a PyTorch alumni roster: Dmytro Dzhulgakov and James Reed both worked on PyTorch core and its compiler at Meta, while Chenyu Zhao arrived from leading Google's Vertex AI platform ⁴. Together AI pursued a parallel strategy under Vipul Ved Prakash, raising an $800 million Series C in 2026 to press the case that open-source inference economics beat closed-model pricing at scale ⁵. Ce Zhang serves as Together's chief technology officer and Tri Dao, known for research underlying faster attention mechanisms, holds the chief scientist title, giving the company a research bench built specifically to squeeze latency and cost out of open-weight serving ⁵.

Groq's leadership shakeup and Cerebras' margin scare

Money and a management change arrived together at Groq this year. The company raised $650 million in June 2026, followed weeks later by a further $350 million round that pushed its valuation to $3.5 billion ⁶. Between those two raises, Groq's leadership changed hands: Adam Winter, who joined the company in 2024 to run its international business, became chief executive in 2026, a transition the company's own leadership page confirms, circumstances left undiscussed ⁶. Winter now oversees a team including chief financial officer Matt Eng, chief operating officer Alan Rice, and chief technology officer Sinclair Schuller, running a company whose custom chip architecture claims to serve "millions of developers" running "trillions of tokens" weekly ⁶. Cerebras took the opposite public journey: Andrew Feldman, the company's chief executive and co-founder, rang Nasdaq's opening bell on May 14, 2026 after a $5.5 billion raise sent shares up 108 percent on debut, only for the stock to fall 10 percent five weeks later when Cerebras forecast a shrinking margin in its first post-IPO earnings report ^{7 8}. The margin warning matters beyond Cerebras specifically: it is public evidence that inference-specialist economics, even for a company running custom silicon rather than reselling Nvidia GPUs, face real pressure from the same price collapse squeezing everyone else's per-token revenue.

Bounded reasoning attacks the cost side directly

Where the inference specialists compete on serving efficiency, one research team is attacking a different lever entirely: how much reasoning a model needs to perform per answer. Armağan Amcalar, chief technology officer at OpenServ Labs, and researcher Eyup Cinar published BRAID, a framework replacing open-ended chain-of-thought reasoning with structured, diagram-encoded logic flows, detailed in "Reasoning's Rebate," this edition's feature examining the paper directly ⁹. The technique's headline result, a performance-per-dollar gain reaching 74 times the baseline on one benchmark, addresses the reasoning-cost problem from the demand side rather than the supply side Fireworks and Together compete on ⁹. Amcalar summarized the ambition in his own words: "BRAID boosts performance across every model class, from largest to smallest, making strong reasoning affordable" ⁹. If techniques like BRAID generalize past benchmark conditions, the Gartner forecast's growth curve could bend, since a meaningful share of that $23.3 billion in 2026 inference spend goes toward reasoning tokens a bounded approach might render unnecessary.

What the spending curve implies for agent infrastructure

Every agent orchestration framework, every payment rail built for machine-to-machine commerce, and every enterprise pilot graduating to production depends on inference remaining affordable enough to run at scale. Model Context Protocol servers and Agent2Agent-connected systems generate inference calls at a volume any single human-triggered workflow historically fell far short of matching, and that volume is precisely what Gartner's forecast is measuring even as sticker price per token keeps falling. Enterprise buyers reading the 96 percent figure in isolation risk sticker shock; read alongside the token-price collapse and the leadership churn among inference specialists racing to serve that demand cheaper, the number reads instead as a market still finding its efficient scale, price and volume locked in a tug-of-war whose outcome stays undetermined for either side.

By the numbers

• $42.276 billion: Gartner's 2026 forecast for worldwide AI-optimized IaaS spending, up 96 percent from 2025 ¹.
• 55 percent: inference's share of that 2026 spending, rising to 59 percent in Gartner's 2027 forecast ¹.
• $89.0 billion: Nvidia's data-center revenue for the quarter ended late July 2026, up 117 percent year over year ².
• 95 percent: reported decline in equivalent-capability inference cost over the two years leading into 2026 ³.
• $17.5 billion: Fireworks AI's valuation after its July 2026 Series D, alongside a reported $1 billion in annual recurring revenue ⁴.
• Two rounds, $650 million then $350 million: Groq's path in 2026 to a $3.5 billion valuation ⁶.
• 108 percent: Cerebras' stock gain on its Nasdaq debut, May 14, 2026, before a margin warning sent shares down 10 percent weeks later ^{7 8}.
• 74 times: peak performance-per-dollar gain BRAID's bounded-reasoning technique reported on one benchmark ⁹.

What to watch

Quarterly earnings from the inference specialists will show whether Cerebras' margin warning was company-specific or an early signal for the entire category, and a second chipmaker or platform reporting compressed margins would confirm the latter. Gartner's 2027 inference-share forecast, rising to 59 percent, deserves a check against real spending data once the year closes, since agent orchestration volume could outpace even that projection. Bounded-reasoning techniques reaching production deployment beyond benchmark conditions would be the clearest sign that the cost curve itself, past the price curve alone, is finally bending toward enterprises running agents at scale.

Sources

1. "Gartner Forecasts Worldwide Artificial Intelligence-Optimized IaaS Spending to Grow 96% in 2026," Gartner, Aug. 10, 2026, https://www.gartner.com/en/newsroom/press-releases/2026-08-10-gartner-forecasts-worldwide-artificial-intelligence-optimized-iaas-spending-to-grow-96-percent-in-2026.
2. "NVIDIA Announces Financial Results for Second Quarter Fiscal 2027," Nvidia, Aug. 26, 2026, https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-second-quarter-fiscal-2027.
3. "How AI Inference Costs Have Dropped 95% in Two Years, and What Happens Next," Value Add VC, 2026, https://valueaddvc.com/blog/how-ai-inference-costs-have-dropped-95-in-two-years-and-what-happens-next.
4. "Announcing our Series D and $1B ARR," Fireworks AI, July 15, 2026, https://fireworks.ai/blog.
5. "Announcing our $800M Series C to accelerate the shift to open-source AI," Together AI, 2026, https://www.together.ai/blog.
6. Groq leadership and news pages, Groq, 2026, https://groq.com/about-us/.
7. "Cerebras raises $5.5B, then stock pops 108%, in the first huge tech IPO of 2026," TechCrunch, May 14, 2026, https://techcrunch.com/2026/05/14/cerebras-raises-5-5b-kicking-off-2026s-ipo-season-with-a-bang/.
8. "Cerebras falls 10% after chipmaker forecasts shrinking margin in first earnings report since IPO," CNBC, June 23, 2026, https://www.cnbc.com/2026/06/23/cerebras-cbrs-q1-earnings-report-2026.html.
9. Armağan Amcalar and Eyup Cinar, "BRAID: Bounded Reasoning for Autonomous Inference and Decisions," arXiv, Dec. 17, 2025, https://arxiv.org/html/2512.15959v1.