Microsoft Maia 300 aims to become its main AI inference chip

Microsoft wants most of the AI answers it serves to come from chips it designs itself. On Monday, October 12, 2026, the company unveiled Maia 300, the third generation of its in-house AI accelerator, and said it will scale the Maia family to multi-gigawatt deployments starting in 2027. In a post on the Official Microsoft Blog, Azure chief Scott Guthrie wrote that the goal is to make Maia "the primary inference system for Microsoft products and services."

That is a big statement from a company whose data centers are still full of Nvidia GPUs. It does not mean Microsoft is breaking up with Nvidia. It does mean that the everyday work of running models for Copilot, Teams, GitHub and Microsoft's own MAI models is supposed to shift onto Microsoft silicon over the next few years. Chief executive Satya Nadella called it "our biggest bet yet on our own silicon" in a post on X, as GeekWire reported.

Below is what Microsoft actually announced, which numbers are company claims, what we can calculate from the published specs, and why the inference market is where this fight gets decided.

The announcement in one paragraph

Maia 300 is an inference-first accelerator built for long-context reasoning, latency-sensitive conversations and multi-step agent work. Production deployments begin in 2027. Microsoft expects it to deliver 40 percent more tokens per megawatt than leading GPU-based systems, and says a full rack of 96 accelerators delivers up to 7x the rack-scale token throughput of a Maia 200 rack. Each chip carries eight HBM4 stacks with 18.0 TB/s of peak memory bandwidth inside a 950 watt TDP, and the compute chiplets are made on TSMC's N2P process. A rack draws 167 kilowatts and holds 27.6 TB of HBM with 1.7 PB/s of aggregate HBM bandwidth. All of these are Microsoft figures. As industry analyst Patrick Moorhead told CNBC: "Nobody has tested them outside of Microsoft yet."

Maia 200 is already doing real work

The Maia 300 announcement lands only nine months after Microsoft introduced Maia 200 in January. That chip is built on TSMC's 3 nanometer process with more than 140 billion transistors, 216 GB of HBM3e at 7 TB/s, 272 MB of on-chip SRAM and a 750 watt TDP. Microsoft said at the time that it offered over 10 petaFLOPS at FP4 and 30 percent better performance per dollar than the latest hardware in its fleet. Nadella repeated the 30 percent figure to analysts in July, according to CNBC.

According to the new blog post, Maia 200 now serves Microsoft production workloads including Microsoft Copilot, and supports the MAI team that builds Microsoft's own models. GitHub Copilot and Microsoft Foundry are scheduled to follow by the end of 2026. Across those workloads the chip runs OpenAI models, Microsoft's MAI models and open model families.

Deployment is spreading too. Microsoft says Maia 200 is rolling out across the United States, Europe and Australia, and claims up to 1.7x more tokens per watt for built-in AI features in Teams, Copilot and Cowork compared with leading GPU infrastructure. GeekWire adds useful context here: as of this summer, Maia 200 was running in just two U.S. data centers, according to The Information in August. So the geographic expansion is recent, and the scale is still small compared with Microsoft's GPU fleet.

What changes from Maia 200 to Maia 300

Microsoft's own framing is that data movement, not raw compute, has become the main bottleneck for large inference systems. Modern reasoning models and agents keep long conversations, documents and tool results in memory, and every new token has to read a large KV cache. That makes inference increasingly memory-bound, especially in the decode phase where the model generates output one token at a time. Maia 300 is designed around that problem.

The published changes:

• Memory bandwidth. Eight HBM4 stacks give 18.0 TB/s per chip, up from 7 TB/s on Maia 200. That is roughly 2.6 times the bandwidth.
• Power. The chip TDP rises from 750 W to 950 W. Bandwidth grows much faster than power.
• Chiplets. Maia 200 was a single near-reticle-sized die. Maia 300 splits compute, memory-controller and I/O functions into separate chiplets. The compute chiplets use TSMC's N2P node. Microsoft's hardware team told The Verge that this "increases the silicon area dedicated to compute performance and allows each function to use the technology best suited to its role."
• Low precision formats. Microsoft highlights an enhanced MXFP4 format to shrink memory and bandwidth needs for both computation and KV cache storage, which in turn allows larger context windows in the same power envelope.
• SRAM as scratchpad. Specialized kernels use on-die SRAM as a software-managed scratchpad to take pressure off HBM.
• Networking. High-bandwidth links between accelerators are tuned for tensor parallelism, and a custom AI transport runs over Ethernet-based infrastructure so the same topology can scale from one rack to a whole data center.

We can also do a little arithmetic with the published numbers. A rack holds 27.6 TB of HBM across 96 accelerators, which works out to roughly 288 GB per chip. Bandwidth per watt goes from about 9.3 GB/s per watt on Maia 200 (7,000 GB/s divided by 750 W) to about 18.9 GB/s per watt on Maia 300 (18,000 GB/s divided by 950 W). That roughly doubles memory bandwidth per watt, which fits Microsoft's claim of "industry leading HBM bandwidth per watt." These are our calculations from Microsoft's numbers, not separately published specs.

The rack is the real product

Microsoft keeps calling Maia a "system," and the rack numbers explain why. A Maia 300 rack packs 96 accelerators, which CNBC reports is double the count in a Maia 200 rack. It runs inside a 167 kilowatt power envelope with 27.6 TB of HBM and 1.7 PB/s of aggregate memory bandwidth in one high-speed scale-up domain. That aggregate matches 96 chips times 18 TB/s.

On that rack, Microsoft claims up to 7x the token throughput of a Maia 200 rack. It is worth pulling that number apart. Doubling the chip count explains about 2x. Faster memory explains a large part of the rest, because decode throughput tends to track memory bandwidth. Newer low precision formats, bigger scale-up domains and software improvements make up the remainder. "Up to" also means the best case workload, not every model in every configuration.

How to read the 40 percent claim

The headline claim is 40 percent more tokens per megawatt compared with leading GPU-based systems. Three things matter when you read it.

1. It is an expectation. The blog says Microsoft "expects" this result. Maia 300 is not in production until 2027, so the figure is based on internal modeling and prototypes. 2. The baseline is vague. "Leading GPU-based systems" is not a named product. By 2027 the leading GPU systems will be Nvidia's Vera Rubin generation and AMD's Helios racks, both of which Microsoft has said it will deploy. Whether the 40 percent is measured against those or against today's hardware is not spelled out in the blog. 3. The metric is the right one. Tokens per megawatt is the number that matters for a cloud operator when power, not floor space, is the limit. As Microsoft general manager Alistair Speirs told CNBC, "You can think of power translating pretty directly into operational cost for something like a data center as well."

It is also interesting that Maia 300's claimed advantage over GPUs (1.4x per megawatt) is smaller than the claimed Maia 200 advantage (up to 1.7x per watt for built-in AI features). That is not necessarily a step backwards. The two numbers use different workloads and very likely different GPU baselines, and GPUs keep getting better too. Treat both as vendor claims until outside benchmarks exist.

Why inference, and why now

Training gets the headlines, but inference is where the money is spent every day. Each Copilot answer, each GitHub coding suggestion, each Teams summary and each agent step costs tokens, and those tokens cost power. Microsoft runs AI features for hundreds of millions of people, so a large efficiency gain on inference adds up fast.

That is why Microsoft is not chasing a general-purpose chip. Training stays with partners. Speirs told CNBC that for training, Microsoft works with Nvidia and AMD. The Maia program is aimed at the part of the stack Microsoft controls completely: its own products and its own models.

Mustafa Suleyman, chief executive of Microsoft AI, made the model side explicit in comments to The Verge: "We have to work across the full stack, all the way down to the silicon. At that scale we need a platform that we own, and that we co-design with our models." Microsoft is trying to catch up with Google DeepMind, OpenAI and Anthropic on frontier models. Owning the inference hardware gives it a way to tune chips and models together, the way Google has done with TPUs for years.

There is also a financial story. Chief financial officer Amy Hood said at a Goldman Sachs conference last month that using its own chips has helped with "margin improvement over the course of a decade," CNBC reported. Every token served on Maia instead of a purchased GPU is a token where Microsoft keeps more of the margin.

Nvidia is still a huge partner

Nothing here ends the Microsoft and Nvidia relationship. Nadella and Nvidia CEO Jensen Huang shared a stage last week to launch Windows PCs built on Nvidia chips, GeekWire notes. Nadella also said on the July earnings call that Microsoft will be among the first clouds to deploy Nvidia's Vera Rubin and AMD's Helios systems.

Microsoft also kept itself room to move. Speirs declined to give a timeline for the multi-gigawatt target: "We reserve a little bit of optionality there, because this is an industry that's changing very fast, and we want to deploy the right silicon for the right models that we're using at the right time as well."

The near term share is modest too. Analysts at Evercore ISI estimate that Maia will represent about 10 percent of Microsoft's AI chip spending by the first quarter of 2027, according to CNBC. That is meaningful money, but it also means about nine in ten AI chip dollars will still go elsewhere next year.

How Maia compares with Google and Amazon

Microsoft is late to custom AI silicon compared with its two biggest cloud rivals. Google has run TPUs for a decade and has begun selling them directly to some customers. Amazon has Inferentia and Trainium and has won over AI labs including Anthropic. Meta builds its own AI chips, and OpenAI unveiled its first custom chip, built with Broadcom, in June, GeekWire reports.

The biggest difference: Maia is not for rent. Unlike Google TPUs or Amazon Trainium and Inferentia, outside companies cannot book Maia capacity by the hour. Maia serves Microsoft's own products and the models Microsoft runs on Azure for customers through services like Foundry. That keeps the program focused, but it also means fewer outside developers write code for it and fewer independent benchmarks will appear.

What we still do not know

• Real benchmark data. No outside lab has tested Maia 300, and the technical paper is a Microsoft document.
• The exact GPU baseline behind the 40 percent tokens per megawatt claim.
• Total power and dates. "Multi-gigawatt" from 2027 has no published year-by-year schedule.
• Price and supply. TSMC N2P capacity and HBM4 supply are both tight across the industry, and Microsoft has not discussed volumes.
• Customer access. Whether Azure customers will ever be able to rent Maia directly, rather than using it indirectly through Microsoft services.

Microsoft says it will share more at Ignite 2026 in San Francisco next month.

Bottom line

Maia 300 is Microsoft's clearest signal yet that it wants to own the economics of AI inference. The published specs show a chip built around memory bandwidth and power efficiency, the rack doubles in density, and Microsoft says the Maia family will become the default engine for its own AI products from 2027. The headline numbers, 40 percent more tokens per megawatt and up to 7x rack throughput, are company claims without outside verification. Nvidia stays in the picture for training and for a large share of inference for some time. But the direction is now official: Microsoft wants its own silicon to answer most of your Copilot prompts.

Sources

• Official Microsoft Blog: Microsoft to expand Azure Maia system to multi-gigawatt deployments
• Microsoft Maia 300 microsite
• Microsoft Source EMEA: Microsoft announces Maia 300
• CNBC: Microsoft to rely more on homegrown AI chips, reducing dependence on Nvidia
• The Verge: Microsoft's new AI chip makes it less reliant on Nvidia's GPUs
• GeekWire: Microsoft makes biggest bet yet on its own silicon
• Official Microsoft Blog, January 2026: Maia 200, the AI accelerator built for inference