Speech to Speech AI Model & Provider Leaderboard

Analysis and comparison of Speech to Speech models & API providers. Artificial Analysis has analyzed speech to speech models and hosting providers across different characteristics including their reasoning quality, conversational dynamics, generation time and price.

For further details, see the methodology page.

Highlights

Speech Reasoning
Speech reasoning based on the Artificial Analysis Big Bench Audio dataset · Higher is better
Speed
Time to first audio (seconds) on Big Bench Audio · Lower is better
Input Price
Price per hour of input audio (USD) · Lower is better

Speech Reasoning

Speech Reasoning (Big Bench Audio)

Speech reasoning: based on the Artificial Analysis Big Bench Audio dataset · Higher is better

Big Bench Audio is an Artificial Analysis benchmark comprising 1,000 audio questions adapted from Big Bench Hard, designed to test the reasoning ability of native audio models. Questions span four categories (250 each): Formal Fallacies (determine whether an argument is deductively valid or invalid), Navigate (determine whether navigation steps return to the starting point), Object Counting (count the number of a specific item class from a list of possessions), and Web of Lies (evaluate the truth value of a Boolean function expressed as a word problem). Models receive an audio input and generate an audio output, which is evaluated as correct or incorrect.

Agentic Performance

Agentic Performance (𝜏-Voice)

Proportion of replica customer service scenarios resolved while acting as a customer support agent, based on the 𝜏-Voice benchmark · Higher is better · Only full duplex models

𝜏-Voice measures task completion in real-world replica customer service scenarios. A voice user simulator calls the tested model with a specific domain problem (e.g., requesting a flight change, disputing a retail charge, resolving a telecom issue) across domains inherited from 𝜏-bench and 𝜏²-bench. The tested model is prompted as a customer support agent with access to domain-specific tools and a policy document. Each task has a single valid database end-state; evaluation compares final state against this ground truth. Charts use average performance across 3 trials where available. For more detail on the original benchmark see the 𝜏-Voice paper.

Note: Following models based on 1 trial: GPT-Realtime-2 (Minimal), OpenAI; Following models based on 2 trials: GPT-Realtime-2 (High), OpenAI

Agentic Performance (𝜏-Voice) by Domain

Proportion of replica customer service scenarios resolved by domain, based on the 𝜏-Voice benchmark · Higher is better · Only full duplex models
Airline
Retail
Telecom

𝜏-Voice measures task completion in real-world replica customer service scenarios. A voice user simulator calls the tested model with a specific domain problem (e.g., requesting a flight change, disputing a retail charge, resolving a telecom issue) across domains inherited from 𝜏-bench and 𝜏²-bench. The tested model is prompted as a customer support agent with access to domain-specific tools and a policy document. Each task has a single valid database end-state; evaluation compares final state against this ground truth. Charts use average performance across 3 trials where available. For more detail on the original benchmark see the 𝜏-Voice paper.

Note: Following models based on 1 trial: GPT-Realtime-2 (Minimal), OpenAI; Following models based on 2 trials: GPT-Realtime-2 (High), OpenAI

Conversational Dynamics

Conversational Dynamics (Full Duplex Bench subset)

Weighted average of pause handling, turn-taking, interruption handling, and backchannel handling from Full Duplex Bench v1 and v1.5 · Higher is better

Metrics from Full Duplex Bench v1 include Pause Handling (% of samples model correctly does not interrupt) and Turn Taking (% of samples correctly takes the turn in conversation). Metrics from Full Duplex Bench v1.5 include User Interruption Handling (% of samples where the model correctly addresses user's interruption) and Backchannel Handling (% of samples where the model correctly continues when a backchannel such as "yeah" or "alright" is played).

Conversational Dynamics (Full Duplex Bench subset) - Category Breakdown

Individual category scores ordered by conversational dynamics score. Based on subset of Full Duplex Bench v1 and v1.5
Pause Handling
Turn Taking
User Interruption Handling
Backchannel Handling

Metrics from Full Duplex Bench v1 include Pause Handling (% of samples model correctly does not interrupt) and Turn Taking (% of samples correctly takes the turn in conversation). Metrics from Full Duplex Bench v1.5 include User Interruption Handling (% of samples where the model correctly addresses user's interruption) and Backchannel Handling (% of samples where the model correctly continues when a backchannel such as "yeah" or "alright" is played).

API Benchmarks

Speech Reasoning (Big Bench Audio) vs. Input Price

Big Bench Audio score vs. input price (USD per hour) · Higher speech reasoning and lower price are better
Most attractive quadrant
Alibaba Cloud
Amazon Bedrock
Google
OpenAI
StepFun
xAI

Price per hour of audio included in the request/message sent to the API, represented as USD per hour of audio.

Big Bench Audio is an Artificial Analysis benchmark comprising 1,000 audio questions adapted from Big Bench Hard, designed to test the reasoning ability of native audio models. Questions span four categories (250 each): Formal Fallacies (determine whether an argument is deductively valid or invalid), Navigate (determine whether navigation steps return to the starting point), Object Counting (count the number of a specific item class from a list of possessions), and Web of Lies (evaluate the truth value of a Boolean function expressed as a word problem). Models receive an audio input and generate an audio output, which is evaluated as correct or incorrect.

Input Price

Price per hour of input audio (USD) · Lower is better

Price per hour of audio included in the request/message sent to the API, represented as USD per hour of audio.

Time to First Audio

Average time to first audio (seconds) on Big Bench Audio dataset · Lower is better

Number of seconds required to generate the first token of audio output. A lower value represents a faster generation.

Summary of Key Metrics & Further Information

Model NameFootnotesSpeech ReasoningConversational DynamicsAgentic PerformanceTime to first audio (secs)Price per hour of audio input (USD)Price per hour of audio output (USD)
StepFun logoStepFun
StepFun logoStep-Audio R1.1 (Realtime)
-
98%--1.510.061.69
Alibaba Cloud logoAlibaba Cloud
Alibaba Cloud logoQwen3.5 Omni Plus Realtime
-
73%--1.260.160.82
Alibaba Cloud logoQwen3.5 Omni Flash Realtime
-
59%--0.790.160.82
Alibaba Cloud logoQwen3 Omni Flash
-
59%72.7%-4.820.611.36
Alibaba Cloud logoQwen3 Omni Realtime
-
57%--0.880.650.99
Hathora logoHathora
Hathora logoQwen3 Omni Flash, Hathora
-
---2.80--
NVIDIA logoNVIDIA
NVIDIA logoNemotron Voicechat
-
39%77.8%----
NVIDIA logoPersonaPlex
-
19%91.0%----
Amazon Bedrock logoAmazon Bedrock
Amazon Bedrock logoNova 2.0 Sonic (Mar 2026)
-
88%--1.140.271.08
Kyutai logoKyutai
Kyutai logoMoshi
-
4%61.0%----
xAI logoxAI
xAI logoGrok Voice Think Fast 1.0
-
97%77.8%52.1%1.25--
xAI logoGrok Voice Agent
-
93%71.6%27.4%0.783.003.00
OpenAI logoOpenAI
OpenAI logoGPT-Realtime-2 (High)
-
97%95.3%38.3%2.331.154.61
OpenAI logoGPT-Realtime-2 (Medium)
-
93%95.2%37.4%1.591.154.61
OpenAI logoGPT Realtime
-
83%93.9%30.4%0.981.154.61
OpenAI logoGPT-Realtime-1.5
-
81%95.7%38.8%0.821.154.61
OpenAI logoGPT-Realtime-2 (Minimal)
-
72%96.1%30.8%1.121.154.61
OpenAI logoGPT-4o mini Realtime (Dec 2024)
-
69%--1.270.361.44
OpenAI logoGPT Realtime Mini (Oct 2025)
-
64%95.7%15.1%0.810.361.44
OpenAI logoGPT-4o audio chatcompletions
54%--3.383.6014.40
OpenAI logoGPT-4o Realtime (Dec 2024)
-
-89.8%27.9%1.491.445.76
Google logoGoogle
Google logoGemini 3.1 Flash Live Preview - High
-
97%-37.7%2.980.351.38
Google logoGemini 2.5 Flash Native Audio Dialog Thinking
-
91%--3.870.351.38
Google logoGemini 3.1 Flash Live Preview - Minimal
-
71%-26.2%0.960.351.38
Google logoGemini 2.5 Flash Native Audio Dialog
-
69%--0.630.351.38
Google logoGemini 2.5 Flash Native Audio Preview (Sep 2025)
-
-30.3%----
Google logoGemini 2.5 Flash Native Audio Preview (Dec 2025)
-
-44.0%22.8%---
VITA logoVITA
VITA logoFreeze-Omni
-
33%58.7%----
Cofe AI logoCofe AI
Cofe AI logoFLM-Audio
-
16%62.0%----

Frequently Asked Questions

Step-Audio R1.1 (Realtime) leads with a speech reasoning score of 97.6% on the Big Bench Audio dataset across 25 models evaluated. Grok Voice Think Fast 1.0 follows at 97.1% and GPT-Realtime-2 (High) at 96.6%.

GPT-Realtime-2 (Minimal) leads with a conversational dynamics score of 96.1% on the Full Duplex Bench dataset across 17 models evaluated. GPT-Realtime-1.5 follows at 95.7% and GPT Realtime Mini (Oct 2025) at 95.7%.

The top speech to speech models by reasoning quality are: 1. Step-Audio R1.1 (Realtime) (97.6%), 2. Grok Voice Think Fast 1.0 (97.1%), 3. GPT-Realtime-2 (High) (96.6%), 4. Gemini 3.1 Flash Live Preview - High (96.6%), 5. Grok Voice Agent (93.3%). Scores are based on the Big Bench Audio benchmark.

The top speech to speech models by conversational dynamics are: 1. GPT-Realtime-2 (Minimal) (96.1%), 2. GPT-Realtime-1.5 (95.7%), 3. GPT Realtime Mini (Oct 2025) (95.7%), 4. GPT-Realtime-2 (High) (95.3%), 5. GPT-Realtime-2 (Medium) (95.2%). Scores are based on the Full Duplex Bench benchmark.

Gemini 2.5 Flash Native Audio Dialog has the lowest Time to First Audio at 0.63s, followed by Grok Voice Agent (0.78s) and Qwen3.5 Omni Flash Realtime (0.79s). Lower values mean faster initial response.

Step-Audio R1.1 (Realtime) is the most affordable at $0.064/hour for input audio, followed by Qwen3.5 Omni Plus Realtime ($0.162/hour) and Qwen3.5 Omni Flash Realtime ($0.162/hour).

Full Duplex Bench evaluates how well speech to speech models handle real conversational behaviors: knowing when to speak, when to stay silent during pauses, how to respond to interruptions, and how to recognize backchannels like "yeah" or "mm-hmm". Artificial Analysis implements a subset of Full Duplex Bench v1 and v1.5.

Speech reasoning (Big Bench Audio) measures whether a model can understand and correctly answer reasoning questions delivered as audio. Conversational dynamics (Full Duplex Bench) measures whether a model can handle natural conversation flow — turn-taking, pauses, and interruptions. A model may excel at one but not the other.

Real-time conversation requires both strong conversational dynamics and low latency. By conversational dynamics score, GPT-Realtime-2 (Minimal) (96.1%), GPT-Realtime-1.5 (95.7%), and GPT Realtime Mini (Oct 2025) (95.7%) lead the field. By Time to First Audio, Gemini 2.5 Flash Native Audio Dialog (0.63s), Grok Voice Agent (0.78s), and Qwen3.5 Omni Flash Realtime (0.79s) are the fastest. The best choice depends on whether natural conversation flow or response speed is more critical for your use case.

Benchmarks are updated regularly as new models and providers are added. Performance metrics are continuously monitored to reflect current provider capabilities and pricing changes.