Google Chirp: API Provider Benchmarking & Analysis
Analysis of Google Chirp API providers across performance metrics including Artificial Analysis Word Error Rate Index, speed, and price.
Highlights
Artificial Analysis Word Error Rate (AA-WER) Index by API
Artificial Analysis Word Error Rate (AA-WER) Index by API
% of words transcribed incorrectly · Lower is better · AA-WER v2 incorporates 3 datasets: AA-AgentTalk (50%), VoxPopuli-Cleaned-AA (25%), Earnings22-Cleaned-AA (25%)
Note: For Earnings22, if a model cannot reliably handle full-length audio due to time limits, we chunk to ~9 minutes (relevant to: Nova 2 Pro, Amazon; Voxtral Mini Transcribe, Mistral; GPT-4o Transcribe, OpenAI; GPT-4o Mini Transcribe, OpenAI). For models with even shorter time limits, we chunk to ~30 seconds (relevant to: Canary Qwen 2.5B, NVIDIA).
API Benchmarks
Artificial Analysis Word Error Rate Index vs. Price
% of words transcribed incorrectly · Lower is better · AA-WER v2 incorporates 3 datasets: AA-AgentTalk (50%), VoxPopuli-Cleaned-AA (25%), Earnings22-Cleaned-AA (25%) · USD per 1000 minutes of audio
Most attractive quadrant
Speed Factor
Speed Factor
Input audio seconds transcribed per second · Higher is better
Price
Price of Transcription
USD per 1000 minutes of audio
Summary of Key Metrics & Further Information
Provider | Further Details | ||||
|---|---|---|---|---|---|
Qwen3.5 Omni Flash |  Alibaba Cloud | 13.5% | 75.5 | 0.00 | |
Qwen3.5 Omni Plus | Alibaba Cloud | 3.5% | 96.9 | 0.00 | |
Nova 2 Pro |  Amazon Bedrock | 4.9% | 22.8 | 3.10 | |
Amazon Transcribe | Amazon Bedrock | 4.1% | 16.2 | 24.00 | |
Universal-3 Pro |  AssemblyAI | 3.1% | 98.6 | 3.50 | |
Universal, AssemblyAI | AssemblyAI | 3.8% | 116.0 | 2.50 | |
MAI-Transcribe-1.5 |  Microsoft Azure | 2.4% | 261.2 | 6.00 | |
MAI-Transcribe-1 | Microsoft Azure | 2.6% | 66.7 | 6.00 | |
transcribe-03-2026 |  Cohere | 4.6% | 39.1 | 0.00 | |
Nova-3 |  Deepgram | 5.2% | 521.2 | 4.30 | |
Nova-2 | Deepgram | 5.3% | 414.9 | 4.30 | |
Base | Deepgram | 10.7% | 310.9 | 12.50 | |
Scribe v2 |  ElevenLabs | 2.2% | 32.7 | 3.67 | |
Scribe v1 | ElevenLabs | 3.0% | 39.5 | 6.67 | |
Solaria-1, Gladia |  Gladia | 4.1% | 59.0 | 4.07 | |
Solaria-3, Gladia | Gladia | 3.2% | 60.3 | 10.16 | |
Gemini 3.1 Pro Preview (High) | Google | 2.8% | 7.1 | 18.15 | |
Gemini 3.1 Pro Preview (Low) | Google | 3.6% | 7.6 | 7.72 | |
Gemini 3 Flash (High) | Google | 2.9% | 16.5 | 13.70 | |
Gemini 2.5 Flash Lite | Google | 5.2% | 62.1 | 6.56 | |
Gemini 2.5 Flash | Google | 5.1% | 70.8 | 6.66 | |
Gemini 2.5 Pro | Google | 2.9% | 13.1 | 11.39 | |
Gemini 3.1 Flash-Lite Preview (Minimal) | Google | 3.4% | 74.7 | 5.83 | |
Gradium Speech-to-Text |  Gradium | 8.4% | 2.2 | 13.00 | |
Grok Speech to Text, SpaceXAI |  SpaceXAI | 4.0% | 104.0 | 1.67 | |
Voxtral Mini Transcribe 2 |  Mistral | 3.6% | 81.0 | 3.00 | |
Voxtral Mini Transcribe | Mistral | 3.5% | 74.4 | 2.00 | |
Voxtral Small | Mistral | 2.8% | 55.0 | 4.00 | |
Voxtral Mini |  DeepInfra | 3.8% | 76.2 | 1.00 | |
Modulate STT Batch English VFast |  Modulate | 4.2% | 54.7 | 0.42 | |
Parakeet TDT 0.6B V3, Togetherai |  Together AI | 4.5% | 970.2 | 1.50 | |
Canary Qwen 2.5B, NVIDIA |  Replicate | 4.3% | 5.7 | 0.74 | |
Parakeet TDT 0.6B V2, NVIDIA |  NVIDIA | 6.4% | 97.1 | 0.00 | |
Parakeet RNNT 1.1B | Replicate | 5.4% | 6.4 | 1.91 | |
GPT-4o Transcribe |  OpenAI | 4.0% | 31.3 | 6.00 | |
GPT-4o Mini Transcribe | OpenAI | 4.5% | 44.7 | 3.00 | |
Smallest AI Pulse Pro |  Smallest.ai | 2.4% | 293.8 | 3.50 | |
Resonant-1 |  Reson8 | 3.4% | 286.8 | 3.60 | |
Rev AI |  Rev AI | 5.9% | 12.4 | 3.33 | |
Smallest AI Pulse | Smallest.ai | 4.4% | 230.6 | 5.00 | |
Soniox v5 Async |  Soniox | 3.8% | 19.6 | 1.66 | |
Soniox V4 | Soniox | 3.9% | 29.3 | 1.66 | |
Speechmatics Standard |  Speechmatics | 5.1% | 76.4 | 4.00 | |
Speechmatics Enhanced | Speechmatics | 4.0% | 59.9 | 6.70 | |
Whisper Large v3 Turbo |  Groq | 4.6% | 121.4 | 0.67 | |
Wizper Large v3 |  fal.ai | 4.7% | 210.5 | 0.50 | |
Incredibly Fast Whisper | Replicate | 5.7% | 56.6 | 1.49 | |
Whisper Large v3 | Replicate | 10.1% | 2.8 | 4.23 | |
Whisper Large v3 | fal.ai | 4.1% | 80.4 | 1.15 | |
Whisper Large v3 | Together AI | 4.5% | 426.6 | 1.50 | |
Whisper Large v2 | OpenAI | 4.1% | 27.2 | 6.00 |