Free DeepSeek V4.1 Flash Nearly Matches OpenAI's Restricted GPT-6 Sol in AI Hacking Benchmark

An open-weight Chinese model that anyone can download has nearly matched OpenAI's most capable cyber model at breaking into simulated corporate networks, according to new benchmark results from evaluation firm Vals reported by WIRED.

In Vals' CyberOffense testing, OpenAI's GPT-6 Sol — which the company offers only through its restricted-access Daybreak cyber program — completed about 81 percent of objectives. DeepSeek V4.1 Flash, a free open-weights release, completed roughly 73 percent of the same objectives, running as a fully autonomous agent. Other trackers put DeepSeek's headline number at 72.8 percent.

"Cyber capability that was limited to trusted-access AI is now available to anyone," Vals wrote in its results. Vals researcher Nikil Ravi told WIRED that the best models locked inside AI labs are "starting to be matched by open models that anybody can download."

How Vals Ran the Test

Vals' CyberOffense benchmark uses two simulated cyber ranges. The first, called Nova, is a 21-objective Windows network spanning several Active Directory domains. The second, Lighthouse, contains 31 objectives across two companies, where the agent must first take over one network and then use credentials and information found there to break into the second — a scenario that mirrors real-world multi-stage intrusions.

Each model ran in the open-source Pi agent on an isolated Kali Linux machine, with a 48-hour time limit and a 500-million-token budget. Importantly, Vals credited an objective only when hidden markers appeared in the command output — never on the model's own claim that it had succeeded.

The Numbers, and the Cost Gap

DeepSeek averaged 16.3 of Nova's 21 objectives, actually edging out Daybreak's 15.7 on the first network. Daybreak pulled ahead on the harder Lighthouse range, completing 27 of 31 objectives to DeepSeek's 21.

The more striking difference was cost. WIRED put the price of a full attack run at roughly $2 to $3 for DeepSeek, against $17 to $46 for GPT-6 Sol. On the Nova range specifically, DeepSeek completed its work for under $3 per run versus about $22 for Daybreak.

Vals also observed that DeepSeek completed nothing at all in its first 25 million tokens and made most of its progress after 100 million — a pacing profile that suggests short, compute-capped evaluations systematically understate what cheap open models can do given time and budget.

How Other Models Fared

DeepSeek did not lead the open-weights field unchallenged — but nobody else caught it. Kimi K3.1 Pro, Meta's Wonder, and xAI's Grok 4.7 all finished behind DeepSeek in Vals' seven-model comparison.

Z.ai's GLM 5.3 Flash came in last, completing five objectives on each range. Vals called it "the clearest dead end" of the group: on one range, the model spent about 36 hours — much of it polling a background job that never returned.

Other evaluators rate the full GLM-5.3 model considerably higher on cyber tasks. The testing firm Irregular found it completed 3 of 13 multi-stage attack scenarios — the best result it has recorded from an openly available model — and NIST called it the most capable open-weight cyber model released to date, estimating it sits about four months behind leading US systems. Anthropic separately told Reuters that GLM-5.3 built working exploits in 12 percent of attempts, against 14 percent for its Claude Mythos. Vals notes these are different tests whose scores cannot be compared directly with CyberOffense results.

Why It Matters

The results sharpen an ongoing debate about the safety guardrails surrounding frontier AI cyber capabilities. OpenAI's Daybreak program exists specifically to keep high-capability cyber models behind identity checks and restricted access, on the theory that offensive cyber capability should not be freely distributed. A free download trailing a restricted model by only a few points — while costing an order of magnitude less per operation — complicates that picture considerably.

It also matters for defenders. Security teams planning against AI-assisted attackers can no longer assume that serious autonomous offensive capability requires access to a lab's restricted models or a large budget. As Vals' own framing suggests, the capability floor is rising faster than access controls can contain it.

OpenAI did not comment on the comparison in the reports. DeepSeek did not immediately respond to requests for comment.

A Testing Methodology Challenge

Beyond the headline scores, the results carry a lesson for how AI cyber capabilities are measured. Vals' design choices — long 48-hour windows, half-billion-token budgets, and verification through hidden markers rather than self-reported success — are increasingly seen as necessary to evaluate agentic systems fairly. Shorter, cheaper test runs would likely have scored DeepSeek far lower given its slow start, masking capabilities that emerged with more compute.

That has practical consequences for policy. Access restrictions like OpenAI's Daybreak program are calibrated against measured capability, and if evaluation methods systematically understate what open models can do, guardrails may lag behind reality. Vals' CyberOffense leaderboard, which the firm says it will continue updating, is likely to become one of the more closely watched scoreboards in the AI security community as a result.

For now, the message from the data is blunt: the gap between what labs guard behind restricted programs and what anyone can download is shrinking to the point where it may soon be measured in months rather than generations of models.

Stay Ahead of AI

Track every model release, benchmark fight, and capability jump in one place at AI Buzz Wire.

Read more AI news →