Qwen 3.8 27B Now Runs at Roughly 1,500 Tokens per Second on Cerebras Inference

Cerebras has added Qwen 3.8 27B to the public endpoints of its Cerebras Inference platform, where the open-weights model runs at roughly 1,500 tokens per second, according to the company's model catalog documentation. The listing makes one of Alibaba's most widely used open model families available at speeds that dwarf typical GPU-hosted serving.

The announcement drew immediate attention from developers: the story gathered more than 600 points on Hacker News within a day, a level of traction that reflects how hot demand for fast, cheap inference has become. For a broader view of the inference market, the breaking AI news desk follows every major platform update as it lands.

The Specs on the Catalog

Per Cerebras' documentation, Qwen 3.8 27B carries 27 billion parameters and supports 64K tokens of context on the free tier and 128K on paid tiers, running at approximately 1,500 tokens per second. It sits alongside OpenAI's open-weight GPT-OSS 120B model, which the same catalog lists at roughly 3,000 tokens per second with 65K context on the free tier and 131K on paid tiers.

Both models are available on Cerebras' free trial and pay-as-you-go tiers, subject to rate limits. Customers needing reserved capacity, higher throughput or production SLAs are directed to the company's Dedicated Endpoints offering, which the documentation also lists as the route to additional model families beyond the two on public endpoints.

The context windows deserve attention alongside the speed figures. Sixty-four thousand tokens on the free tier — and 128,000 on paid — is enough to hold sizeable codebases, long documents or extended agent transcripts in memory at once, which matters for the exact workloads where fast decoding pays off. Cerebras' documentation also includes an OpenAI compatibility guide, lowering the switching cost for teams whose existing code already targets the OpenAI SDK: in principle, pointing an existing client at a Cerebras endpoint is a configuration change rather than a rewrite.

Unpruned Models, Transparent Compression

One detail worth noting in the documentation is Cerebras' disclosure of how it handles model compression. The company states that all models on its public endpoints are original, unpruned versions, and that it uses selective weight-only quantization only during storage to preserve quality. Sensitive layers stay at full precision, activations, attention and the KV cache remain unquantized, and dequantization happens on the fly.

Cerebras also discloses its research into pruning techniques such as REAP (Router-weighted Expert Activation Pruning): pruned variants are shared with the research community on Hugging Face but are not served through the public API. For developers choosing where to run open models, that transparency about what exactly is being served is unusually explicit.

Why Token Speed Matters

Throughput numbers like these matter most for agentic and coding workloads. Applications that chain hundreds of model calls — coding agents, autonomous workflows, real-time assistants — multiply per-token latency across every step, so the difference between 50 and 1,500 tokens per second compounds into a qualitatively different experience. Interactive applications that stream long completions benefit most visibly.

The trade-off is scope. A 27-billion-parameter model will not match frontier systems on the hardest reasoning tasks, and Cerebras' own docs steer heavy production workloads toward dedicated capacity. But for the large middle ground of tasks where mid-sized open models are already good enough — summarization, classification, tool use, code editing — speed becomes the differentiator.

There is also a price dimension developers will weigh. Public endpoint models are usable on a free trial before any commitment, which makes benchmarking against a team's own workload essentially friction-free — a deliberate on-ramp that contrasts with enterprise contracts that require sales conversations before the first token is served. The documented rate limits are the constraint to watch: free-tier access exists to prove the speed, not to run production traffic.

A Busy Stretch for Open Models

The addition lands during a crowded stretch for open-weights AI. UAE-based lab IFM just introduced K2 Horizon, a connected fleet of six fully open models, and Meta continues to iterate its Muse family for coding and agentic use. Meanwhile, open-model ecosystems in China keep shipping at a pace that has compressed release cycles across the industry.

For developers, the practical takeaway is that the open-model stack is maturing on two fronts at once: capability, as mid-sized models close gaps with flagships on everyday tasks, and serving economics, as specialized inference providers compete on raw speed. Qwen 3.8 27B on Cerebras is a data point for both trends — a current-generation open model served at speeds that were exotic a year ago, on hardware purpose-built for the job.

Developers evaluating the platform should benchmark their own workloads: published speeds are catalog figures, real-world throughput depends on prompt lengths, concurrency and configuration, and the free tier's rate limits will bind before its speed does.

Stay Ahead of AI

Every model release, inference platform update and developer tool launch, tracked as it happens — read more AI news →