Clef-Flash: Release, Decision Index Score, Performance & Price

Model summary

46
Artificial Analysis Decision Index
1 out of 4 units for Decision Index.

Time

298 ms
Median time per task
4 out of 4 units for Time.

Cost

$32
Cost per 1M tasks (USD)
2 out of 4 units for Cost.

Score

Artificial Analysis Decision Index

Weighted average of AA-Omniscience-Decision, HLE-MultiChoice, When2Call and WildGuardTest · Higher is better

Artificial Analysis Decision Index vs. Cost per 1M Tasks

Artificial Analysis Decision Index score vs. average cost per 1 million tasks (USD) · Lower is better
Most attractive quadrant
Pareto line

Calibration

Artificial Analysis Decision Index Calibration

Accuracy against the probability the model gave its chosen answer · Bins of 0.1 with at least 20 answers · Each benchmark weighted equally

Calibration for language models coming soon.

Time

Artificial Analysis Decision Index: Time per Task

Median time per task over every task across all Artificial Analysis Decision Index evaluations · Lower is better

Cost

Artificial Analysis Decision Index: Cost per 1M Tasks

Average cost per 1 million tasks across all Artificial Analysis Decision Index evaluations at current prices (USD) · Lower is better