Thinking Machines Lab, the research company founded by former OpenAI chief technology officer Mira Murati, has published a detailed technical explanation of how it trains large language models almost entirely in 4-bit precision — a technique the company says unlocks major speed and memory savings on Nvidia's newest data center hardware.
The blog post, titled "End-to-End 4-Bit Training" and published on the company's website on November 13, walks through the numerical trade-offs behind low-precision training and describes how the team managed them while training its Inkling 2 Small model. For readers following the latest AI developments, the post offers a rare public look at the engineering decisions that determine how much a frontier-scale training run actually costs.
Why Fewer Bits Means Faster Training
Training a large language model involves calculating and storing enormous quantities of numbers. Using fewer bits for each value reduces memory usage and lets GPUs perform arithmetic faster. The cost is precision: fewer bits mean fewer representable numbers, so values must be rounded more aggressively.
The company illustrates the problem with a simple example. If a format can only represent the numbers 3 and 4, storing 3.2 requires rounding to 3, introducing a quantization error of 0.2. A single rounding error is trivial, but during training these errors accumulate across layers and optimization steps, and they can interfere with the learning process itself.
"This is the central challenge of low-precision training: how can we benefit from faster arithmetic without introducing errors that interfere with learning?" the company wrote in the post.
Three Formats, One Training Run
Rather than committing to a single precision level, Thinking Machines uses three numerical formats at different stages of training: BF16, MXFP8, and NVFP4. Each format has fixed precision, but its speed depends on the hardware running it.
The company trains on Nvidia's Grace Blackwell Ultra platform, the GB300, which is specifically designed to accelerate matrix multiplications in NVFP4. According to the post, NVFP4 stores values with 3.6 times fewer bits than BF16 while running up to 6 times faster on GB300 systems. In raw throughput terms, the company lists GB300 performance at roughly 15 PFLOPS in NVFP4, 5 PFLOPS in MXFP8, and 2.5 PFLOPS in BF16.
The catch is that NVFP4's aggressive compression introduces far more quantization error than the wider formats. That error is acceptable in some stages of training and unacceptable in others, which is why the company treats each stage separately.
Different Stages, Different Rules
The blog post breaks training into four stages — the forward and backward passes of pretraining, the reinforcement learning sampler, and the RL trainer — and assigns each a different balance of priorities between minimizing error, minimizing bias, increasing speed, and reducing memory use.
Speed matters most in the high-volume steps: pretraining, which runs over trillions of tokens, and RL sampling, which generates large volumes of rollouts. Backward passes, however, are a special case. The post explains that backward passes average over many tokens in a batch, which dilutes the impact of randomly distributed rounding errors — but they are highly sensitive to quantization bias, the systematic drift that occurs when values consistently round in one direction. Unlike random noise, that bias can compound across layers, samples, and training steps.
The company says it managed these trade-offs while training Inkling 2 Small, the open-weight mixture-of-experts model it released earlier this month under an Apache 2.0 license. The post does not claim that every part of the run ran in 4 bits; rather, it describes an end-to-end strategy in which the precision of each component is chosen deliberately rather than inherited by default.
Why Low-Precision Training Matters
The economics here are straightforward. Compute is the dominant cost in frontier model development, and hardware vendors have been racing to add low-precision acceleration to each new chip generation. Nvidia has positioned NVFP4 support in its Blackwell Ultra platform as a headline feature, and rival accelerators from AMD, Google, and others advertise their own reduced-precision formats.
If research labs can safely push more of the training pipeline into 4-bit arithmetic, the same hardware budget buys more experiments, more tokens, or larger models. That is a meaningful advantage in a market where access to cutting-edge accelerators is scarce and expensive. It also compounds with the industry's broader push toward efficiency techniques such as quantization-aware inference and speculative decoding, both of which squeeze more capability out of fixed hardware.
The post is also a signal of where the research community is heading. Low-precision training was long viewed as too risky for anything beyond inference, because rounding errors were assumed to corrupt learning. Careful stage-by-stage analysis — like the one Thinking Machines published — is how that assumption gets tested and, where possible, overturned.
From Blog Post to Open Weights
What makes the disclosure notable is that it is not purely theoretical. The company tied the technique directly to a model it has actually shipped. Inkling 2 Small was released with downloadable weights in early November, and the new post explains the numerical strategy behind its training. Practitioners who download the model are, in effect, getting a working artifact of the low-precision methodology the company describes.
Thinking Machines has been unusually open for a frontier-scale lab, publishing technical blog posts alongside its model releases. That transparency gives outside researchers something rare: a documented, reproducible account of how a modern training run handles the messy numerical realities of mixed-precision arithmetic.
For the broader industry, the message is that 4-bit training has moved from research curiosity to production technique. As more labs adopt similar strategies, the gap between theoretical hardware peak performance and realized training throughput will narrow — and the cost of training capable models will continue to fall.
Stay Ahead of AI
The techniques behind frontier models evolve weekly. Follow AI Buzz Wire for clear, sourced reporting on the research that shapes the field.
Read more AI news →
