The economics of AI infrastructure have flipped. Roughly two-thirds of the workloads running through AI infrastructure today are inference rather than training, according to Positron AI executive Cameron McCaskill, who shared the figure in an interview with theCUBE Research at the SC26 supercomputing conference. Three years ago, the split was almost the exact opposite: about 85 percent training and 15 percent inference.

That reversal is quietly redrawing the map of where AI compute lives — and it explains why a new generation of inference-first chipmakers is suddenly winning cloud deals from companies far larger than themselves.

The Numbers Behind the Flip

"We're at a point where about two-thirds of the workloads being run through these AI infrastructures are inference versus one-third is training," said McCaskill, head of sales and business development at Positron, in the interview with theCUBE Research's Rebecca Knight and John Furrier. "Back in 2023, it was like 85% training, 15% inference. So now you can see why inference is driving the economic decisions of how they're going to build out their infrastructure."

The shift makes intuitive sense. Training happens at a handful of frontier labs with billion-dollar clusters. Inference happens everywhere, every time a user prompts a model — and as consumer and enterprise adoption has exploded, the aggregate demand for tokens has overwhelmed the infrastructure built for the training era. Power, cooling, and memory supply now shape where tokens get generated as much as raw chip performance does, and that changes which hardware wins deals.

Positron's Bet on Commodity Memory

Positron AI, which raised $875 million at a $5 billion valuation in September, has built its entire strategy around the inference era. The company's central design choice: skip high-bandwidth memory (HBM), the scarce and expensive memory type that anchors most AI accelerators, and build inference appliances around commodity memory instead.

"Everyone else pretty much in the industry is using what's called HBM or high bandwidth memory. It's very limited in supply, very expensive," McCaskill said. "We went with commodity memory."

The trade-offs are deliberate. Commodity memory is cheaper and far easier to source, and Positron's architecture keeps power low enough for air-cooled rooms — no exotic liquid cooling required. A single Positron server draws about 4,500 watts, and a full rack of the company's upcoming Titan servers fits within 40 kilowatts.

The roadmap is advancing quickly. Positron's first-generation Atlas server ran on a programmable chip, and the company plans to tape out its own silicon early next year for the Titan server. The company has already landed marquee deployments: Oracle is bringing roughly 340 Atlas servers live in its cloud this quarter, according to McCaskill, and a neocloud provider has signed on for Atlas while committing to the upcoming Titan generation.

Modular Data Centers Become the Natural Home for Inference

Perhaps the most consequential part of the interview is where McCaskill says the demand is coming from: not hyperscale campuses, but small, fast, modular facilities.

"One of the groups of companies that we're seeing a rise in right now and coming to us for product is these modular data centers, where they have a half megawatt or, 1 megawatt or 2.5 megawatt modular data center, and those can be deployed very quickly," he said. "The goal of a lot of these inference players is I want to get from land and power to monetizable tokens within 90 to 120 days."

That timeline — roughly one quarter from breaking ground to revenue — is impossible for a gigawatt-scale training campus, which can take years to permit, build, and energize. Inference workloads are smaller, more geographically flexible, and tolerant of sites where operators can find a few megawatts of cheap power without heavy regulatory hurdles. The result is an emerging tier of distributed inference infrastructure sitting in telco sites, colocation facilities, and modular units closer to where demand actually is.

What It Means for the Industry

For chip vendors, the message is that the AI hardware market is bifurcating. Training continues to reward the most advanced accelerators and the deepest pockets. Inference rewards different virtues: power efficiency, memory cost, deployment speed, and the ability to run in ordinary air-cooled buildings. Vendors optimized only for the training race may find the largest volume of future deployment decisions — where tokens are actually monetized — happening on hardware designed for a different set of constraints.

For enterprises and cloud customers, the shift is mostly invisible but increasingly consequential: inference capacity that arrives in 90-to-120-day increments means more competitive pricing and more geographic options for running models near users and data.

And for the AI industry's buildout narrative — often framed as an endless race for gigawatt campuses — the inference flip is a reminder that the demand curve is broadening, not just scaling. The two-thirds milestone cited at SC26 is one executive's estimate, but it matches the direction every major cloud operator has reported: the AI buildout's center of gravity is moving from training the models to serving them.

Stay Ahead of AI

Track the latest AI developments as they happen with AI Buzz Wire, your source for AI hardware and infrastructure news.

Read more AI news →