Microsoft on Monday introduced Azure Maia 300, the third generation of its in-house AI accelerator family, and said the chip will anchor multi-gigawatt deployments across Azure starting in 2027 — a major step in the company's effort to run more of its AI workloads on its own silicon rather than on Nvidia's GPUs.
The announcement, made in a blog post by Scott Guthrie, Microsoft's executive vice president of its Cloud + AI group, positions Maia 300 as the primary inference system for Microsoft's own products and services. It lands amid a broader industry race to build custom AI chips, a shift we track closely in our ongoing coverage of the latest AI developments as hyperscalers work to tame the cost of serving AI at scale.
According to Microsoft, Maia 300 is designed to deliver 40% more tokens per megawatt compared with leading GPU-based systems. In a full 96-accelerator configuration, the company says the chip provides up to seven times the rack-scale token throughput of its predecessor, Maia 200.
Built for Inference From the Ground Up
Microsoft describes Maia 300 as an "inference-first" architecture aimed at long-context, memory-intensive and decode-heavy workloads — the patterns that dominate modern chat assistants, reasoning systems and multi-step AI agents.
The chip's memory subsystem uses eight stacks of HBM4 delivering 18.0 terabytes per second of peak bandwidth inside a 950-watt thermal envelope. On-die SRAM is treated as a software-managed scratchpad, which Microsoft says eases pressure on HBM and keeps more data close to the compute units.
Maia 300 also moves to a multi-chiplet design that splits compute, memory-controller and I/O functions into separate dies, with the compute chiplets fabricated on TSMC's N2P process node. Support for enhanced MXFP4 low-precision formats is intended to shrink both compute memory needs and KV-cache storage, allowing larger context windows and higher throughput within the same power budget.
On the networking side, Microsoft treats the network as an extension of the chip's dataflow architecture. A custom AI transport provides scale-up communication across accelerators and racks over Ethernet-based infrastructure, forming what the company describes as a unified topology that scales from a single rack to datacenter-scale deployments.
Rack-Scale Numbers
At rack level, Microsoft says 96 Maia 300 accelerators operate within a 167-kilowatt power envelope, offering 27.6 terabytes of HBM capacity and 1.7 petabytes per second of aggregate bandwidth across the scale-up domain. The company argues that this density lets it serve more simultaneous AI interactions and larger workloads from each rack.
Maia 200 Is Already in Production
Maia 300 follows Maia 200, which Microsoft introduced in January 2026 and which already serves production workloads, including Microsoft Copilot. According to the company, Maia 200 will also underpin GitHub Copilot and Microsoft Foundry by the end of 2026, and it runs OpenAI models, Microsoft's own MAI models and leading open-source model families.
Microsoft says Maia 200 is deploying across the United States, Europe and Australia, delivering up to 1.7 times more tokens per watt for built-in AI features in Teams, Copilot and Microsoft 365 experiences compared with the leading GPU infrastructure it replaced. Guthrie wrote that the operational experience from Maia 200 directly shaped the Maia 300 design.
Together, the two generations are intended to power a wide set of Microsoft AI services: multi-step agentic work behind GitHub coding tools, cloud-scale model serving through Microsoft Foundry, inference and data generation for the MAI model team, and latency-sensitive Dragon Copilot workflows that aim to reduce clinicians' administrative burden.
Reducing Reliance on Nvidia
The strategic subtext of the announcement is Microsoft's effort to reduce its dependence on Nvidia, whose GPUs dominate AI training and inference across the industry. Reuters reported that Microsoft plans a "huge deployment" of its own chips as a way to rein in costs, while CNBC framed the rollout as a direct move to cut the company's reliance on Nvidia hardware — some of the most expensive and supply-constrained components in the AI buildout.
Nvidia's pricing power has become a boardroom issue across the cloud industry, and custom silicon is the standard response. Google's TPUs, Amazon's Trainium and Inferentia lines, and Meta's MTIA chips all pursue the same goal: specialized hardware tuned to each company's exact workloads. Microsoft's Maia program, which began with the Maia 100 unveiled in November 2023, is its entry in that race.
For Microsoft, the economics are straightforward: every token served by an in-house chip is a token that does not carry Nvidia's margin. With AI inference volumes growing across Copilot, Microsoft 365, GitHub and Azure AI services, even partial substitution translates into substantial savings at hyperscale.
What Comes Next
Microsoft said production deployments of Maia 300 will begin in 2027, and pointed readers to a published technical paper for architectural detail. The company also said it will share more about its silicon strategy and its broader infrastructure roadmap at Microsoft Ignite 2026, which it is hosting in San Francisco and online.
Whether Maia 300 can dent Nvidia's dominance remains to be seen — Nvidia's latest architectures still lead on raw performance, and Microsoft continues to buy Nvidia hardware at enormous scale. But the trajectory is clear: the world's largest cloud operator intends to serve an ever-larger share of the world's AI tokens on chips it designed itself.
---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
