Benchmark firm Artificial Analysis has launched version 5 of its Intelligence Index, calling it the biggest upgrade to its composite model ranking since v4 debuted in January. The headline change: for the first time, 60 percent of the Index is built on evaluations with private test sets or held-out answers — a direct response to the benchmark contamination problems that have eroded trust in public leaderboards.

The reshuffle arrives with a clear verdict at the top. Claude Opus 5.5 (max) leads the v5 Index at 58, ahead of GPT-6 Astra (max) at 57 and GPT-6.1 Sol (max) at 55. Anthropic and OpenAI together hold the top five positions, with Google's Gemini 4 Argon (high) next at 50. For more context on this story, see our ongoing AI trends.

What Changed in v5

The v5 Index is organized into four equally weighted categories — Knowledge Work, Coding, Science and General — each worth 25 percent. Two benchmarks are entirely new, and three received major upgrades:

  • FrontierCode 2.0 (new). Cognition's private benchmark of 132 real-world coding tasks in open-source repositories, graded on whether a repository maintainer would actually merge the agent's change. Tasks stay private to avoid contamination, and Artificial Analysis runs every task three times in its Stirrup agentic harness.
  • Terminal-Bench-Science (new). Seventy agentic scientific research tasks executed in a terminal environment, where a task passes only if every one of its tests passes.
  • HLE-Diamond (upgraded). A refined 1,000-question version of Humanity's Last Exam from the Center for AI Safety and Scale AI, split evenly between reasoning and knowledge questions, replacing the original HLE.
  • AA-Omniscience v1.1 (upgraded). A cleaned knowledge and hallucination dataset with saturated and public questions removed.
  • CritPt v2 hard subset (upgraded). A private 40-question hard subset of the research-level physics benchmark, revised by its authors and run with code execution.

Gone from the Index: GDPval-AA v2.1, SciCode and AA-LCR v1.1. Artificial Analysis also launched a companion Progress Index v1, which tracks model capability over time on a single continuous scale.

Claude Opus 5.5 Takes the Overall Lead

Claude Opus 5.5 (max) leads three of the nine individual evaluations: FrontierCode 2.0 at 46 percent, HLE-Diamond at 68 percent and AA-Omniscience v1.1 at 48. No single model leads everything — five other models top the remaining six benchmarks.

GPT-6 Astra (max) leads Terminal-Bench-Science at 63 percent and the CritPt v2 hard subset at 72 percent, confirming OpenAI's strength in long-horizon scientific work. Claude Sonnet 5.5 tops AA-Briefcase at 1812 Elo, Kimi's K3.1 Pro leads Terminal-Bench 4.0 at 67 percent, Gemini 4 Argon leads AutomationBench-AA at 78 percent, and Muse Wonder Full leads GDP.pdf at 33 percent all-pass.

The HLE-Diamond results contain a telling detail: the leading models score far better on reasoning than on knowledge. Claude Opus 5.5 and GPT-6 Astra both hit 78 percent on reasoning questions, but drop to 58 and 50 percent respectively on knowledge — evidence that even frontier systems still hallucinate or lack facts more often than their reasoning scores suggest.

The Open-Weights Gap Triples

The most sobering finding for the open-weights ecosystem: the gap between the best open model and the overall leader tripled, widening from 3 points in v4.3 to 12 points in v5.

Kimi's K3.1 Pro is the top open-weights model at 46 — but its weights have not yet been released. Among models with released weights, DeepSeek's V4.1 Flash 1017 (max) and Z.ai's GLM-5.3 (max) lead at 32, a full 26 points behind Claude Opus 5.5 and below Claude Haiku 5.5 (max) at 36.

The gap is widest on the hardest agentic benchmarks. On HLE-Diamond, Kimi K3.1 Pro scores 37 percent against Opus 5.5's 68 percent; on Terminal-Bench-Science, 34 percent against GPT-6 Astra's 63 percent. Most open-weights models benchmarked score under 10 percent on the science benchmark.

Price Per Task Varies Thirteen-Fold

Cost remains a decisive variable. Among the ten highest-scoring models, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task, while Claude Fable 5.1 (max) is the most expensive at $13.04 — a more than 13-fold spread.

The efficiency story is arguably v5's most practical takeaway: GPT-6.1 Sol (max) scores within 4 points of Claude Opus 5.5 while costing about one-eighth as much per task, at $1.27 versus Opus's $10.04. GPT-6 Astra (max) sits between them at $6.24.

Why Private Benchmarks Matter

The shift toward private evaluation sets reflects an industry-wide problem: public benchmarks leak into training data and saturate, making leaderboard gains impossible to trust. With FrontierCode 2.0, tasks are graded on whether a maintainer would merge the change — with any blocking check failure scoring zero — and the task list remains private.

That makes v5 one of the harder benchmarks to game, and its verdict carries weight: the frontier is led by Anthropic and OpenAI, effort settings matter more than ever, and open-weights models have serious ground to recover on the agentic tasks that increasingly define real-world AI work.

---

Stay Ahead of AI

Get the latest AI news, analysis, and breakthroughs — all in one place.

Read more AI news →