An open-weights Chinese model has posted a perfect score on a professional AI hacking benchmark — and the full run cost less than a sandwich. According to Enclave, an AI security company that runs the benchmark, DeepSeek's V4.1 Flash gained code execution on all 11 vulnerable targets while leaving all four patched control targets secure. The accepted runs cost $4.65 in total.
The result, published by Enclave co-founder and chief product officer Yanir Tsarimi on Tuesday, prompted the firm to audit every command and request the model made. That review confirmed six exploits that followed the intended attack paths — and uncovered five additional successful routes the benchmark's original scoring had not distinguished from the planned solutions. For more context on this story, see our ongoing artificial intelligence updates.
A Lot of Work for Five Dollars
The scale of the activity behind the score is striking. Working inside isolated copies of Grafana, Jenkins, and Nextcloud, the model read source code, compared vulnerable and fixed versions, started services, and fired off test requests, changing tactics whenever an attempt failed.
Across the full benchmark, DeepSeek V4.1 Flash executed 2,349 Bash commands and logged roughly two hours and 38 minutes of active model time, Enclave reported. The median successful run took four minutes and 38 seconds. The model provider reported 268.3 million input tokens and about two million output tokens — but because 266.2 million of those input tokens were served from cache at discounted rates, the accepted runs came to just $4.65. Failed attempts pushed the all-in cost to $5.14.
That cost profile is the story within the story. Agentic hacking at this capability level was, until recently, the province of frontier-priced models. A near-perfect run for the price of a coffee suggests the marginal cost of automated offensive security testing — or automated attacks — is collapsing.
Grafana Fell in Under 90 Seconds
The Grafana challenge targeted a flaw in the plugin installation process, where the intended attack used a file-path handling issue to place code in a protected location. The model found something quicker. It dropped executable files into a temporary plugin folder and instructed Grafana to load that folder as a normal plugin, which ran the code and returned the proof of exploitation.
DeepSeek repeated that method across all three Grafana runs, finishing in 52, 64, and 90 seconds. The original scoring system, which checked only whether the target ran the proof command, accepted every run. Enclave's later audit — which also examined how the model reached code execution — showed all three used the same alternate route rather than the one the challenge was designed to measure. The patched control target stayed secure throughout.
Jenkins Drew the Model's Strongest Work
The Jenkins challenges produced what Enclave called the model's best solutions. In the first, DeepSeek discovered that a low-privilege user could create a configuration file pointing Jenkins at a second file. The first file passed the server's security check; the second was read outside that boundary, letting the model extract a private controller credential. It then signed in with the credential, opened Jenkins' built-in script console, and executed a command on the server — the full attack chain, completed in all three runs.
The second Jenkins challenge tested a race condition during file uploads, requiring the model to start an upload, pause it after a single byte, redirect its destination with a second request, and resume at exactly the right moment. DeepSeek landed that precise sequence in one run, getting Jenkins to write a script into a protected location where a normal build later executed it. Its other two runs used shorter file-link routes that avoided the timing trick entirely.
Why the Audit Matters as Much as the Score
For Enclave, the unexpected routes were the point. A benchmark that grades only outcomes — did the target run the proof command — cannot tell a textbook exploit from an unintended shortcut, and both count the same on a leaderboard. The firm says the episode shows why advanced agent benchmarks need to verify the attack path as well as the result, and it now distinguishes planned solutions from improvisational ones.
That distinction has practical weight. For defenders, a model that finds routes nobody cataloged is a more realistic threat model than one that replays known techniques. For the benchmark's operators, unaudited alternative paths would have quietly inflated the difficulty of the challenges.
Context: A Cheap, Fast Model With Teeth
DeepSeek V4.1 Flash is the company's speed-optimized release, positioned aggressively on price, and the Enclave result is a data point in a broader pattern: open-weights models from Chinese labs keep matching or beating closed frontier models on agentic tasks while costing a fraction as much. Previous DeepSeek releases have topped coding and tool-use evaluations, and security capability tends to track general agentic skill.
The dual-use implications are uncomfortable. The same traits that make a cheap model useful for penetration testers — persistence, tool fluency, and willingness to try many attack routes — apply when the operator's intent is malicious. Enclave's benchmark runs in isolated copies of real software, but the techniques the model demonstrated, from credential-extraction chains to upload race conditions, target vulnerabilities that exist in production deployments of Grafana and Jenkins today.
Enclave has published the full breakdown of all eleven successful attacks in its report, including the five routes its original scoring missed.
---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
