DualEntry benchmark: open models average 83.9%, closed 81.3%

DualEntry, an AI-native ERP company, published the results on October 26 through Business Wire, with the full leaderboard on its benchmark site. The test, called DualBench100, runs each model through 101 synthetic accounting cases across eight areas: transaction classification, journal entries, financial reporting, bank reconciliation, accounts payable, accounts receivable, month-end close, and accounting knowledge. The cases split into 37 easy, 36 medium, and 28 hard. Every model runs the full set three times under the same instructions and tools inside DualEntry's test accounting environment, and a task counts as passed when the model passes it in at least two of the three runs, with an 80% pass mark. DualEntry says all models were run under its evaluation release as of October 13, 2026.

This briefing covered Moonshot AI's Kimi K3.1 Pro launch on October 23. What is new here is a measurement of it by a third party that is not Moonshot: DualEntry's round puts Kimi K3.1 Pro at 86.1%, fourth overall, ahead of every model from Anthropic, OpenAI, and Meta in this round, according to DualEntry. These are not Moonshot's launch numbers, but DualEntry is a software vendor with its own commercial interests, not an independent academic evaluator.

Google's Gemini 4 Argon leads at 91.1%. Tied for second at 88.1% are Zhipu AI's GLM-5.3 Flash, an open model, and Grok 4.7, which the leaderboard lists under SpaceXAI. Kimi K3.1 Pro follows at 86.1%. Behind it sit Anthropic's Claude Fable 5.1 at 85.1%, then two open models, DeepSeek's V4.1 Flash and Moonshot's earlier Kimi K3, at 84.2% each. Meta's Muse Wonder and Google's Gemini 3.8 Flash follow at 83.2% each, and Anthropic's Claude Opus 5.5 and Zhipu's GLM-5.3 at 82.2% each. OpenAI's GPT-6 Astra and GPT-6.1 Sol are at 81.2% each. The leaderboard lists 21 models in total. One caveat on the labels: DualEntry lists Kimi K3.1 Pro as open, but Moonshot said at launch on October 23 that its full weights are promised by November 4, so it counts in the open group before the weights are public.

DualEntry reports that the seven open models average 83.9% while the 13 closed models average 81.3%, and that open models hold four of the top seven places. Those are simple averages of each group's model scores (both reproduce from the leaderboard scores); one listed model, Mercury 3 from Inception Labs, has no license type on the leaderboard and is outside both groups. The 2.6-point gap is small and DualEntry reports no significance test for it; its methodology says it uses paired comparisons rather than treating every small percentage-point gap as meaningful. Kimi K3.1 Pro's strongest result is bank reconciliation, where it scores 100% against 75% for the overall leader, Gemini 4 Argon. On cost, DualEntry notes that DeepSeek's V4.1 Flash scores 84.2% at $0.003 per task, while Claude Fable 5.1 scores 85.1% at $1.18 per task, roughly 400 times the cost for a score less than one point higher.

DualEntry says accuracy is not the whole story. On hard tasks, the top score of 82.1% is shared by three closed models, Gemini 4 Argon, Grok 4.7, and Gemini 3.8 Flash. On consistency, the five models that most often pass a task in all three runs are all closed, led by Gemini 4 Argon at 86.1%, against 76.2% for GLM-5.3 Flash, the best-scoring open model. Consistency here means the share of tasks a model passes in all three runs, so a model can pass a task in two of three runs, which counts as a pass overall, without being consistent.

Benedict Dohmen of DualEntry said in the release: “Finance teams have assumed that choosing an open model meant trading accuracy for cost and control. On accounting work that trade is mostly gone. What’s left is consistency, and that is the gap to watch.” That is the vendor’s interpretation of its own results. The hardest areas in this round were accounts receivable and month-end close, where models averaged 64.7%, against 93.5% on transaction classification. Every model scored 100% on the multiple-choice accounting knowledge questions, so that category does not separate the models.

Keep the provenance straight. This is a vendor benchmark from an ERP company that sells AI accounting software, not an academic study, and the cases are synthetic, not real client books. No customer data was used, models worked from a freshly restored test company, and every answer was saved as a draft, never posted. DualEntry says scoring is deterministic against fixed reference answers, with no AI model grading another model's work, so the same response always gets the same score. DualEntry also says the results describe the full tested configuration, meaning the model together with DualEntry's tools and instructions, not the model in isolation.

But a pass here means the response met the checks defined for that case, not that every accounting detail was perfect, and DualEntry says so on its own methodology page. Treat the ranking as evidence about this kind of structured accounting work under these conditions, not as a verdict on which model is best in general. A narrower takeaway for readers choosing a model: on this benchmark, open models scored close to closed ones on accuracy and behind them on consistency and hard tasks, so evaluating an open model alongside closed ones on your own cases is reasonable.