Open-weight AI models are closing in on their closed frontier rivals on real-world accounting work, according to the latest results from the Accounting AI Benchmark run by AI accounting company DualEntry — with Google's Gemini 4 Argon leading overall and Chinese open models posting near-identical accuracy for a small fraction of the price.
The benchmark, covered by Yahoo Finance under the headline that open-source AI models are "catching up" to closed ones on accounting work, tests leading models on 101 tasks across eight areas of accounting, from transaction classification and journal entries to reconciliation, financial reporting, and month-end close. Every model works from the same company books, receives the same instructions, and has access to the same accounting tools, with each model run three times to measure consistency. For more context on this story, see our ongoing breaking AI news.
Gemini 4 Argon Leads, but the Gap Is Thin
Gemini 4 Argon tops the leaderboard at 91.1% accuracy, completing tasks in an average of 18 seconds at a cost of $0.115 per task, with the best consistency score of any model at 86.1%.
But the chase pack is close — and remarkably cheap. Z.ai's open-weight GLM-5.3 Flash scored 88.1%, tied with xAI's Grok 4.7, while costing just $0.006 per task, roughly 5% of what Google's flagship model spends. Kimi K3.1 Pro, another open-weight model from Moonshot AI, followed at 86.1%, putting two open models among the top four performers.
Anthropic's Claude Fable 5.1 posted 85.1% at $1.18 per task — the most expensive model tested — while Claude Opus 5.5 scored 82.2% at $0.452. OpenAI's GPT-6 Astra reached 81.2% at $0.494 per task.
The Cost Story Is the Real Headline
The starkest number in the results may be DeepSeek's. DeepSeek V4.1 Flash scored 84.2% at $0.003 per task, according to DualEntry's key takeaways — compared with Claude Fable 5.1 at 85.1% and $1.18 per task. That is roughly 400 times the cost for a score less than one point higher.
Cost per task varied enormously even among models with similar accuracy, the benchmark's authors note, suggesting that enterprises automating bookkeeping workflows could cut inference bills by an order of magnitude simply by choosing the right model for the task.
Where Models Still Struggle
Accounting is not a solved problem for any frontier lab. Performance varied sharply by task area: models averaged 93.5% on transaction classification but just 64.7% on the two hardest areas, accounts receivable and month-end close. The widest spread appeared in accounts receivable, where scores ranged from 41.7% to 91.7%.
No single model dominated. Kimi K3.1 Pro beat Gemini 4 Argon on bank reconciliation, scoring a perfect 100% against Google's 75%, while Kimi K3 led month-end close at 91.7%. Gemini 3.8 Flash was the only model to post a perfect score on journal entries. Every model tested aced the benchmark's accounting-knowledge multiple-choice questions, scoring 100%.
Consistency also diverged from raw accuracy: Grok 4.7 tied for second on accuracy at 88.1% but completed all three runs successfully on only 74.3% of tasks, while Claude Fable 5.1 was more consistent at 80.2% despite a lower accuracy score.
A Confession of Interest — and a Signal for Enterprise AI
DualEntry, which builds AI-native accounting software, publishes the benchmark to showcase the model landscape for its own domain, so enterprises should treat it as one data point rather than a neutral court of final appeal. Its methodology is at least transparent: identical books, identical tools, identical instructions, three runs per model, with results, costs, and durations published per task.
Still, the pattern echoes what independent evaluations have shown across other domains: the open-weight tier — led by Z.ai's GLM family, Moonshot's Kimi models, and DeepSeek — now sits within a few percentage points of the closed frontier on structured professional work, at costs that make volume deployment trivial. For accounting firms and finance teams evaluating AI agents, the frontier label may no longer be worth the premium.
The Details Behind the Rankings
The per-task breakdown shows how differently the models fail. GLM-5.3 Flash matched Gemini 4 Argon on financial reporting at 100% and scored 92.3% on both transaction classification and journal entries, but fell to 75% on bank reconciliation and accounts receivable. Gemini 4 Argon, by contrast, posted perfect scores on transaction classification and financial reporting and led accounts receivable at 91.7% — yet managed only 75% on bank reconciliation, the task Kimi K3.1 Pro aced.
That inconsistency across task types matters for practitioners. A finance team automating invoice processing faces a very different model choice than one automating month-end close, and the benchmark's area-level data suggests the "best" model depends on which ledger entries dominate the workload.
Why Accounting Is a Useful Test Bed
Accounting has emerged as a favored proving ground for agentic AI because it is structured, verifiable, and unforgiving: a journal entry either balances or it does not. Unlike chatbot arenas that reward fluent prose, task-based benchmarks in this domain measure whether a model can follow multi-step instructions, use external tools correctly, and avoid fabricating numbers — the same capabilities enterprises need for back-office automation generally.
The page describes its results as a "current round," with all models evaluated under the same benchmark release, and the round already includes twenty models spanning US and Chinese labs. If the cost curve in this round holds — frontier accuracy at commodity prices — the practical gap between open and closed models on professional knowledge work may soon be measured in dollars rather than points.
---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
