Google Gemini

Google Gemini: API Provider Benchmarking & Analysis

Name: WER Index (Non-streaming)
Creator: Artificial Analysis
License: https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf

Creator:Google

License:Proprietary

Visit

Analysis of Google Gemini API providers across performance metrics including Artificial Analysis Word Error Rate Index, speed, and price.

Highlights

WER Index (Non-streaming)

AA-WER v2 · % of words transcribed incorrectly · Lower is better

Speed Factor

Input audio seconds transcribed per second · Higher is better

Price

USD per 1000 minutes of audio · Lower is better

Artificial Analysis Word Error Rate (AA-WER) Index by API

% of words transcribed incorrectly · Lower is better · AA-WER v2 incorporates 3 datasets: AA-AgentTalk (50%), VoxPopuli-Cleaned-AA (25%), Earnings22-Cleaned-AA (25%)

Note: For Earnings22, if a model cannot reliably handle full-length audio due to time limits, we chunk to ~9 minutes (relevant to: Nova 2 Pro, Amazon; Gemini 2.5 Lite, Google; Voxtral Mini Transcribe, Mistral; GPT-4o Transcribe, OpenAI; GPT-4o Mini Transcribe, OpenAI; Gemini 2.0 Flash Lite, Google). For models with even shorter time limits, we chunk to ~30 seconds (relevant to: Canary Qwen 2.5B, NVIDIA).

Measures transcription accuracy across 3 datasets to evaluate models in real-world speech with diverse accents, domain-specific language, and challenging channel & acoustic conditions.

AA-WER is calculated as an audio-duration-weighted average of WER across ~8 hours from three datasets: AA-AgentTalk (50%), VoxPopuli-Cleaned-AA (25%), and Earnings22-Cleaned-AA (25%). See methodology for more detail.

API Benchmarks

Artificial Analysis Word Error Rate Index vs. Price

% of words transcribed incorrectly · Lower is better · AA-WER v2 incorporates 3 datasets: AA-AgentTalk (50%), VoxPopuli-Cleaned-AA (25%), Earnings22-Cleaned-AA (25%) · USD per 1000 minutes of audio

Most attractive quadrant

Amazon Transcribe

Canary Qwen 2.5B, Replicate

Enhanced

Gemini 2.0 Flash Lite

Gemini 2.5 Flash

Gemini 2.5 Flash Lite

Gemini 2.5 Pro

Gemini 3 Flash (High)

Gemini 3.1 Flash-Lite Preview (Minimal)

Gemini 3.1 Pro Preview (High)

Gemini 3.1 Pro Preview (Low)

GPT-4o Mini Transcribe

GPT-4o Transcribe

Grok Speech to Text, xAI

MAI-Transcribe-1

MAI-Transcribe-1.5

Nova 2 Pro

Nova-3

Parakeet TDT 0.6B V3, Togetherai

Rev AI

Scribe v2

Smallest AI Pulse

Smallest AI Pulse Pro

Solaria-1

Universal

Universal-3 Pro

Voxtral Mini Transcribe

Voxtral Mini Transcribe 2

Voxtral Small

Whisper (L, v3), fal.ai

Whisper (L, v3), Fireworks

Whisper Large v3, together.ai

Wizper (L, v3), fal.ai

Measures transcription accuracy across 3 datasets to evaluate models in real-world speech with diverse accents, domain-specific language, and challenging channel & acoustic conditions.

Estimated cost in USD to transcribe 1,000 minutes of audio, normalized across providers with different billing models, and including billed reasoning tokens where available. Further detail on the methodology page.

Speed Factor

Input audio seconds transcribed per second · Higher is better

Audio file seconds transcribed per second of processing time. Higher factor indicates faster transcription speed. Reported Speed Factor values are medians across benchmark trials from the last 7 days; over-time chart points are daily medians. Artificial Analysis measurements are based on an audio duration of 10 minutes. Speed Factor may vary for other durations, particularly very short durations under 1 minute.

Price

Price of Transcription

USD per 1000 minutes of audio · Lower is better

Summary of Key Metrics & Further Information

	Provider	Whisper Version				Further Details
Qwen3.5 Omni Flash	Alibaba Cloud		13.5%	79.8	0.00	Details
Qwen3.5 Omni Plus	Alibaba Cloud		3.5%	93.7	0.00	Details
Nova 2 Pro	Amazon Bedrock		4.9%	22.7	3.10	Details
Amazon Transcribe	Amazon Bedrock		4.1%	19.3	24.00	Details
Universal-3 Pro	AssemblyAI		3.1%	90.2	3.50	Details
Universal, AssemblyAI	AssemblyAI		3.8%	112.5	2.50	Details
MAI-Transcribe-1.5	Microsoft Azure		2.4%	269.6	6.00	Details
MAI-Transcribe-1	Microsoft Azure		2.6%	55.6	6.00	Details
Nova-3	Deepgram		5.2%	472.0	4.30	Details
Nova-2	Deepgram		5.3%	428.6	4.30	Details
Base	Deepgram		10.7%	330.3	12.50	Details
Scribe v2	ElevenLabs		2.2%	37.4	3.67	Details
Scribe v1	ElevenLabs		3.0%	39.8	6.67	Details
Solaria-1, Gladia	Gladia		4.1%	61.3	4.07	Details
Gemini 3.1 Pro Preview (High)	Google		2.8%	6.3	18.15	Details
Gemini 3.1 Pro Preview (Low)	Google		3.6%	7.1	7.72	Details
Gemini 3 Flash (High)	Google		2.9%	18.2	13.70	Details
Gemini 2.5 Flash Lite	Google		5.2%	70.3	6.56	Details
Gemini 2.5 Flash	Google		5.1%	62.2	6.66	Details
Gemini 2.5 Pro	Google		2.9%	13.7	11.39	Details
Gemini 2.0 Flash Lite	Google		3.8%	56.4	0.19	Details
Gemini 3.1 Flash-Lite Preview (Minimal)	Google		3.4%	70.6	5.83	Details
Gradium Speech-to-Text	Gradium		8.4%	2.3	13.00	Details
Grok Speech to Text, xAI	xAI		4.0%	102.6	1.67	Details
Voxtral Mini Transcribe 2	Mistral		3.6%	70.7	3.00	Details
Voxtral Mini Transcribe	Mistral		3.5%	61.3	2.00	Details
Voxtral Small	Mistral		2.8%	67.1	4.00	Details
Voxtral Mini	DeepInfra		3.8%	70.2	1.00	Details
Modulate STT Batch English VFast	Modulate		13.0%	193.6	0.42	Details
Parakeet TDT 0.6B V3, Togetherai	Together.ai		4.5%	919.9	1.50	Details
Canary Qwen 2.5B, NVIDIA	Replicate		4.3%	5.3	0.74	Details
Parakeet TDT 0.6B V2, NVIDIA	NVIDIA		6.4%	101.9	0.00	Details
Parakeet RNNT 1.1B	Replicate		5.4%	5.9	1.91	Details
GPT-4o Transcribe	OpenAI		4.0%	31.7	6.00	Details
GPT-4o Mini Transcribe	OpenAI		4.5%	46.0	3.00	Details
Smallest AI Pulse Pro	Smallest.ai		3.3%	233.9	5.00	Details
Rev AI	Rev AI		5.9%	12.6	3.33	Details
Smallest AI Pulse	Smallest.ai		4.4%	138.9	5.00	Details
Speechmatics Standard	Speechmatics		5.1%	68.1	4.00	Details
Speechmatics Enhanced	Speechmatics		4.0%	58.2	6.70	Details
Whisper Large v3 Turbo	Groq	v3 Turbo	4.6%	108.4	0.67	Details
Whisper Large v3 Turbo	Fireworks	v3 Turbo	4.7%	213.0	1.00	Details
Wizper Large v3	fal.ai	large-v3	4.7%	232.0	0.50	Details
Incredibly Fast Whisper	Replicate	large-v3	5.7%	56.5	1.49	Details
Whisper Large v3	Replicate	large-v3	10.1%	2.8	4.23	Details
Whisper Large v3	fal.ai	large-v3	4.1%	79.7	1.15	Details
Whisper Large v3	Fireworks	large-v3	4.6%	300.3	1.00	Details
Whisper Large v3	Together.ai	large-v3	4.5%	393.9	1.50	Details
Whisper Large v2	OpenAI	large-v2	4.1%	27.1	6.00	Details

Speech to Text providers compared: Google.