Chinese AI lab Z.ai has published a detailed technical account of how it set GLM models to work improving the very systems that serve them, describing an "Infra Agent" that finds and fixes performance bottlenecks in the company's inference infrastructure with limited human intervention.
The blog post, titled "Toward Recursive Self-Improvement: How GLM Built Its Own Inference Infrastructure," went up on the company's site on Wednesday and quickly became one of the most-discussed technology stories on Hacker News, where it drew more than 200 points from the developer community. For more context on this story, see our ongoing AI industry coverage.
"As we develop GLM, the model sometimes exhibits capabilities that surprise us, and even unsettle us," the company wrote in the post's opening lines.
From Cybersecurity Research to Self-Optimizing Systems
The work traces back to October 2025, when Z.ai began researching how to strengthen GLM's cybersecurity capabilities. The reasoning, the company says, was straightforward: cybersecurity is a natural extension of coding, and models that can navigate large codebases can eventually navigate the low-level systems code that most engineers avoid.
That foundation evolved into something more ambitious. Rather than using models merely to write code, Z.ai set out to answer a systems engineering question: can an AI agent take responsibility for the performance of a production inference stack, the software and hardware coordination layer that turns model weights into fast, cheap responses?
The company's answer involved reframing the problem. End-to-end metrics alone, the post argues, cannot guide an optimizing agent. A report that time-to-first-token increased by 30 percent or that output throughput dropped by 20 percent tells an agent that something got worse, but not which layer is responsible, why its hypothesis failed, or what to test next.
"How do we turn sparse end-to-end results into fine-grained, attributable engineering feedback that directly guides the next action?" the company asks in the post, describing the question as key to the entire effort.
Three Rules for Agent-Readable Feedback
Z.ai's answer is a feedback discipline built on three characteristics. First, feedback must be sufficiently local, tied to specific engine launch parameters, code changes, kernels, input conditions, threads, execution intervals, or code paths, so the agent can narrow the scope of any problem. Second, it must be inexpensive and timely to obtain: a question answerable by a kernel test or a local microbenchmark should not require a full service deployment. Third, validation should match the current hypothesis, with every experiment designed to answer a specific question rather than follow a fixed sequence.
The company says this dense feedback loop is what allows the Infra Agent to behave less like an autocomplete for engineers and more like a colleague who can run experiments overnight.
Three Case Studies, From Numerics to a Threading Bug
The post walks through three concrete episodes from the optimization of the company's serving stack for GLM-5.3-Flash, the open-weight model Z.ai released under the MIT license in August.
The first involves correctness. Because inference optimizations reshuffle how computations are parallelized, the agent needed to verify that kernels produce the same results under partitioned and unpartitioned conditions. In one case, the team combined three TF32 Tensor Core operations to produce a higher-precision result, reducing accumulated numerical error while preserving most of the Tensor Core performance advantage. Comparative testing between partitioning strategies exposed the discrepancies that motivated the fix.
The second is a systems-debugging story that will feel familiar to any performance engineer. Acceptance criteria required that adding KV cache transfer to the serving path should not degrade prefill performance by more than 5 percent under identical workloads. The agent found the gap exceeded 20 percent in some scenarios, which narrowed the investigation to the concurrency behavior of KV transfer itself. Examining execution timelines, the agent identified an anomaly: in the problem scenarios, Python-side KV transfer execution never overlapped with the dispatch and combine intervals of the DeepEP communication library. The fix was to release Python's global interpreter lock during the relevant C++ execution intervals, allowing the transfer thread serving Mooncake, the KV cache store, to proceed in parallel with computation. The performance gap subsequently fell within the acceptance threshold.
The third episode shows iterative kernel tuning. Introducing ReplaySSM, a technique that trades compute for memory, initially increased execution time of a KDA decode kernel. The agent's division optimization then reduced the kernel's execution time by 9.6 percent. After receiving feedback that computation remained the primary bottleneck, the agent discovered the original implementation tiled along the V dimension, causing identical FP32 normalization and gating computations to be repeated unnecessarily.
The Stack Behind GLM-5.3-Flash
The optimizations sit on top of an aggressive serving architecture that Z.ai describes in the post: intra-node tensor parallelism for linear attention and the language model head, the ReplaySSM compute-for-memory trade, W8A8 quantization, mixed-precision cache quantization using INT8, FP8 and BF16 formats, a Layer Split scheme, and an Encode-Prefill-Decode disaggregated architecture that runs the serving stages on separate resources.
The company frames the accumulation of these techniques, discovered and validated largely by the agent itself, as a step toward recursive self-improvement: AI systems improving the infrastructure that makes them faster and cheaper to run, which in turn makes the next round of agent-driven improvement more affordable.
Why It Matters
Inference cost is one of the decisive economics of the AI industry. Labs that can serve frontier-class models at lower cost per token can price aggressively, subsidize agents that make thousands of calls per task, and fund further training runs from serving revenue. Z.ai has been notably aggressive on all three fronts, and the company's interim financial disclosures earlier this year showed revenue growing sharply even as losses narrowed.
The post is also a signal about where lab-side engineering is heading. The traditional division of labor, with human performance engineers tuning kernels and schedulers while models handle user requests, is being replaced at Z.ai by a loop in which the model proposes changes, cheap experiments validate them, and dense feedback directs the next attempt.
Claims of fully autonomous self-improvement should be read with care. The post itself makes clear that human engineers defined the test scenarios, set the acceptance criteria, and built the feedback environments the agent operates within. What the company describes is automation of a well-instrumented optimization loop, not an unsupervised system redesigning itself.
Even so, the direction is clear. If the economics of AI in the next few years are decided by who can serve intelligence cheapest, the labs that teach their models to optimize their own infrastructure may set the pace, and Z.ai wants the industry to know it intends to be one of them.
---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
