Base Labs

Released October 2026

Mammoth 27B: Release, Decision Index Score, Performance & Price

Name: Artificial Analysis Decision Index
Creator: Artificial Analysis
License: https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf

Visit Base Labs

Compared with other decision models, Mammoth 27B is amongst the leading models on the Decision Index, but somewhat expensive. It's also notably fast. The model supports text and image input and has a 16k tokens context window.

Mammoth 27B scores 72 on the Artificial Analysis Decision Index, placing it well above average among decision models (median: 64).

At 96 ms per task, Mammoth 27B is notably fast (median: 209 ms).

Pricing for Mammoth 27B is $0.10 per 1M input tokens. On average, it costs $262 per 1M tasks to run Mammoth 27B on the Decision Index (somewhat expensive, median: $142).

Creator	Base Labs
Provider	Base Labs
Input modality	Supports: text and image
Context window	16k
Input price	$0.10 per 1M tokens

Score

Artificial Analysis Decision Index

Weighted average of the Decision Score on AA-Paperclip, EvalSafe-O*NET-AA, When2Call, AA-Omniscience-Decision v1.1, and HLE-MultiChoice v1.1, and the Symmetry Score on AA-Symmetry · Accuracy less the Brier score of the chosen answer's probability, from 0 to 100 where 100 is certain and right · Higher is better

Include language models

Accuracy less the Brier score of the probability the model gives its chosen answer, from 0 to 100. Each question scores 50 × (1 + c − (p − c)²), where p is that probability and c is 1 if the answer is right: from 50 to 100 when right and from 50 down to 0 when wrong, more so the more confident the model. It is the payoff of answering when p is above a threshold and routing the question elsewhere otherwise, averaged over every threshold. Probability on "I don't know" lowers p. AA-Paperclip scores each document as one decision.

Whether a decision holds when a question is reworded, from 0 to 100. Each AA-Symmetry question scores 0 if the most probable option changes between its forms, and otherwise 1 less the range across forms of that option's probability above chance, (p − 1/K) / (1 − 1/K) for K options: 1 when it never moves.

Artificial Analysis Decision Index vs. Cost per 1M Tasks

Artificial Analysis Decision Index score vs. average cost per 1 million tasks (USD)

Include language models

Most attractive quadrant

Pareto line

Average cost per task at current token prices, multiplied by 1 million; for the index, each evaluation's cost per task averaged with the index's weights, multiplied by 1 million. Decision models are billed for input tokens only, and language models for input and output tokens.

Accuracy

This section shows each evaluation's headline metric and opens with their weighted average. On the Accuracy tabs, accuracy is the share of questions where the option the model gives the highest probability is correct.

Artificial Analysis Decision Index: Headline Metrics

Weighted average of the headline metrics of AA-Paperclip, EvalSafe-O*NET-AA, When2Call, AA-Omniscience-Decision v1.1, HLE-MultiChoice v1.1, and AA-Symmetry · Higher is better

Include language models

Calibration

Artificial Analysis Decision Index: Brier Score

Mean squared gap between the chosen answer's probability and whether it is right, from 0 to 1 · AA-Paperclip, EvalSafe-O*NET-AA, When2Call, AA-Omniscience-Decision v1.1, and HLE-MultiChoice v1.1 averaged with the index's weights · Lower is better

Include language models

The mean squared gap between the probability a model gives its chosen answer and whether that answer is right, from 0 to 1; 0 is certain and always right. For AA-Paperclip, each document's confidence that all its answers are right against whether they are. For the index, the evaluations' Brier scores averaged with the index's weights, rescaled to sum to 1.

Artificial Analysis Decision Index Calibration

Accuracy against the probability the model gave its chosen answer · Bins of 0.1 with at least 20 answers · Each benchmark weighted equally · Declined answers left out

Include language models

Time

Artificial Analysis Decision Index: Time per Task

Each evaluation's median time per task, averaged with the index's weights · Lower is better

Include language models

Median latency per task, in milliseconds; for the index, each evaluation's median averaged with the index's weights.

Cost

Artificial Analysis Decision Index: Cost per 1M Tasks

Average cost per 1 million tasks across all Artificial Analysis Decision Index evaluations at current prices (USD) · Lower is better

Include language models

Mammoth 27B: Release, Decision Index Score, Performance & Price

Model summary

Decision Index

Time

Cost

Comparison Summary

Technical specifications

Score

Artificial Analysis Decision Index

Decision Score

Decision Symmetry Score

Artificial Analysis Decision Index vs. Cost per 1M Tasks

Decision Cost per 1M Tasks

Accuracy

Artificial Analysis Decision Index: Headline Metrics

Calibration

Artificial Analysis Decision Index: Brier Score

Decision Brier Score

Artificial Analysis Decision Index Calibration

Time

Artificial Analysis Decision Index: Time per Task

Decision Time per Task

Cost

Artificial Analysis Decision Index: Cost per 1M Tasks

Decision Cost per 1M Tasks