Skip to content
D-CSIL™

AI News · 2026-11-21 · 1:03 PM CT

76: a new top score for decision models

Paul Derrington, Jr.

By Paul Derrington, Jr.

CEO | Chief Engineer | Research Engineer | Editor, D-CSIL

2 min read

Follow @PaulDerrington on X

Key Takeaways

Baseten released two open-weight decision models built on Google’s Gemma 4 on November 20. Bison-1 scored 75.88 (rounded to 76), topping Artificial Analysis’s Decision Index v2.1 when checked November 21. Baseten sells API access and says both models accept images and up to 131,072 input tokens per question.

An American bison, the animal Baseten's bison-1 decision model is named for
Photo: Kurt Kaiser / Wikimedia Commons (CC0)

What 76 means

A decision model returns probabilities for fixed choices instead of writing a free-form answer. Artificial Analysis’s Decision Index v2.1 combines five evaluations scored for correctness and confidence with a separate symmetry test that checks how consistent answers are across equivalent question forms.

On the leaderboard checked November 21, Bison-1 scored 75.88, ahead of Liquid AI’s d1.2-ultra (74.85) and Baseten’s Mammoth-1 (71.36). Baseten reported a field of 19 at launch. Bison-1 leads the overall index, not every individual evaluation.

The second number: 0.14

Artificial Analysis reports Bison-1 with the lowest aggregate Brier score among the decision models checked (0.1411, versus 0.1499 for d1.2-ultra). The metric averages the squared gap between the model’s confidence in its chosen answer and whether that answer was right. Lower is better, but it does not guarantee that a particular threshold will work for a different dataset.

Baseten attributes the results in part to probability-based training targets, also called soft labels, and says both models were trained on 6.71 million decision examples with an option to abstain. The training figures are Baseten’s own and were not independently verified.

Two models, eight times the previous API limit

Baseten describes Bison-1 as a dense 31B model post-trained from Gemma 4 31B. Groundhog-1 uses Gemma 4 26B-A4B, a mixture-of-experts model with roughly 4B active parameters. Baseten lists a limit of 131,072 tokens per question for both models in its API, up from 16,384 for the preceding models. This is an input limit, not an accuracy guarantee.

Baseten reports its own LongMemEval M and S results at 128k tokens: 84.8 and 87.1 for Bison-1, and 77.4 and 79.3 for Groundhog-1. These company-run results were not independently verified, and Groundhog-1 did not have an Artificial Analysis Decision Index score when checked.

Price, weights and the open question

Baseten says its API charges $0.10 per million input tokens for Bison-1 and $0.035 for Groundhog-1. The company also provides downloadable weights; the Gemma 4 base terms and the license on Baseten’s modifications are listed in the model cards.

The index measures how well models answer a defined set of questions. It does not test whether those questions are the right ones for a particular workflow, or what happens when a system acts on the answers.

Disclaimer: Derrington Collaborative AI is my personal business, operated independently of and outside my official U.S. Government position and duties. The views and opinions expressed on this site and on any and all of my social media accounts are my own. They do not reflect the official policy or position of the Department of the Army, the Department of Defense, or the U.S. Government. Mention of any company, product, or service does not constitute an official endorsement.