•

Open weights model

•

Released julho de 2024

Análise de inteligência, desempenho e preço do Llama 3.1 Instruct 405B

Resumo do modelo

7*
Artificial Analysis Intelligence Index

* Estimado

1 de 4 unidades para Inteligência.

Velocidade

N/D
Tokens de saída por segundo
Valor desconhecido entre 4 unidades para Velocidade.
Entrada -Saída -
N/D
Custo por tarefa do Intelligence Index
Valor desconhecido entre 4 unidades para Custo.

Verbosidade

N/D
Tokens de saída do Intelligence Index
Valor desconhecido entre 4 unidades para Verbosidade.

Artificial Analysis Intelligence Index · Higher is better

Velocidade

Output tokens per second · Higher is better
Weighted average cost (USD) per Intelligence Index task · Lower is better

Inteligência

Artificial Analysis Intelligence Index

Artificial Analysis Intelligence Index v4.3.2 incorporates 10 evaluations: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, Terminal-Bench 4.0, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience, AA-LCR v1.1
No publicly available API
Estimate (independent evaluation forthcoming)

Artificial Analysis Intelligence Index: pesos abertos vs. proprietários

Artificial Analysis Intelligence Index v4.3.2 incorporates 10 evaluations: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, Terminal-Bench 4.0, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience, AA-LCR v1.1
No publicly available API
Estimate (independent evaluation forthcoming)

Intelligence Index vs. custo por tarefa do Intelligence Index

Artificial Analysis Intelligence Index · Weighted average cost (USD) per Artificial Analysis Intelligence Index task
Most attractive quadrant
Pareto line

Intelligence Evaluations

Intelligence evaluations measured independently by Artificial Analysis · Higher is better
Ver mais
No publicly available API

Agentic knowledge work, (Elo-500)/2000

Agentic SaaS workflows

Agentic real-world work tasks, (Elo-500)/2000

Reasoning & knowledge

Professional document reasoning, All-pass

Agentic coding & terminal use

Long context reasoning

SciCodeUnder review

Coding

CritPtUnder review

Physics reasoning

Agentic scientific research workflows in a terminal

Agentic software engineering in open-source repositories

Quantitative analysis on spreadsheets & documents

Legal agentic work, Hallucination-Gated All-Pass Rate

Agentic business operations

Kubernetes incident root-cause analysis

Medical long context reasoning

Visual reasoning

AA-Omniscience

AA-Omniscience Index

AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. Scores range from -100 to 100, where 0 means as many correct as incorrect answers, and negative scores mean more incorrect than correct.
No publicly available API

Openness Index

Artificial Analysis Openness Index: pontuação

O Openness Index avalia a abertura dos modelos em uma escala normalizada de 0 a 100 (quanto maior, mais aberto)

Comparações do Intelligence Index

Intelligence Index vs. custo por tarefa do Intelligence Index

Artificial Analysis Intelligence Index · Weighted average cost (USD) per Artificial Analysis Intelligence Index task
Most attractive quadrant
Pareto line

Custo

Custo por tarefa do Intelligence Index

Weighted average cost (USD) per Intelligence Index task · Segmented by token type · Lower is better

Custo de executar o Artificial Analysis Intelligence Index

Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index

Preços: acertos de cache, entrada e saída

Price (USD per M Tokens)

Janela de contexto

Janela de contexto

Context window: tokens limit · Higher is better
No publicly available API

Tamanho do modelo (somente modelos de pesos abertos)

Tamanho do modelo: parâmetros totais e ativos

Comparison between total model parameters and parameters active during inference (billions)

Perguntas frequentes

Perguntas comuns sobre o Llama 3.1 Instruct 405B