Text to Speech Benchmarking Methodology
Scope & Background
Artificial Analysis performs benchmarking on Text to Speech models delivered via serverless API endpoints. This page describes our Text to Speech benchmarking methodology, including both our quality benchmarking and performance benchmarking. We consider Text to Speech endpoints to be serverless when customers only pay for usage, not a fixed rate for access.
For both our performance benchmarking and within the Speech Arena, our focus is reflecting the end-user experience of users using the serverless APIs. We focus on benchmarking the time to receive the audio file locally. Where the API response is a URL rather than bytes, we include the time of downloading the file in our response time measurement. Our approach is to use the standard implementation of provider APIs as suggested by each provider's documentation. Where on option on the provider's API, we standardize the sample rate of audio to 22.05 kHz.
Key Metrics
We use the following metrics to track quality, performance and price for Text to Speech models.
Quality ELO
Relative ELO score of the models as determined by responses from users in the Artificial Analysis Text to Speech Arena.
Some models may not be shown due to not yet having enough votes. We use a similar Linear Regression model, similar to how LMSYS calculates ELO scores for Chatbot Arena.
Price per 1M Characters
For providers which do not provide a price per 1M characters, we have estimated pricing based on the following alternative methodologies.
For providers which charge based on inference time, we have estimated pricing based on their inference time using a dataset of ~25 texts of ~500 characters. This methodology has been applied for Replicate & fal.ai.
For providers which only offer subscription plans, we select the plan priced closest to $300 per month, which is usually representative of 'Scaled' used of the API, and we assume 80% utilization of the characters offered by that plan. For example, if a $300 plan includes 1 million characters we assume 800,000 characters are used and the price per 1M characters is (300/(1*80%) = $375 per 1M characters). This methodology has been applied for ElevenLabs, Cartesia and LMNT.
Note: when reporting price, we do not include temporary discounts.
Generation Time
Median time the provider takes to generate a single audio clip with ~500 input characters, calculated over the past 14 days of measurements.
Generation Time includes downloading the audio clip from the provider where a URL is provided rather than an audio response. This is to reflect the end-user latency of receiving a generated audio clip and as URLs can be generated prior to audio completion. Audio clips are generated at batch size of 1 where relevant.
Benchmarking is conducted 4 times daily at random times each day. For each benchmarking evlauation we select a single random voice for each model. A unique prompt of ~500 characters is used for each generation.
Model Voices
For each model tested, we test multiple voices to ensure that our comparison between models is representative and fair. Voice characteristics such as accent, gender and style are typically aspects of the voices that each model can generate speech for, not the underlying model. For each model we select 2 voices of each combination of Male and Female, and US and UK accents (8 combinations in total). Where a gender and accent is not available, we exclude this combination from evaluation in the Speech Arena.
Voices are selected for each model based on their prominence in provider interface and documentation, excluding voices which are not neutral in nature (e.g. Los Angeles valley and deep southern accents are excluded). Creators of the models may also request we use specific voices where many are available. Where voices are not provided, as is typically the case for open source models, we use voices clips from professional voice actors as source files for generating speech. All voice clips have been licensed for commercial use.
Below, we list the voices used for each model.
| Model Name | Model Creator | Voices Used (Gender, Accent) |
|---|---|---|
| Arcana |
| Luna (F, US), Orion (M, US), Ursa (M, US), Andromeda (F, US) |
| AsyncFlow V2 |
| Faith (F, US), Penelope (F, US), Griffin (M, US), Xavier (M, US), Robert (M, UK), Chase (M, UK), Nyomi (F, UK), Violet (F, UK) |
| Azure Neural |
| Andrew Multilingual (M, US), Brian Multilingual (M, US), Emma Multilingual (F, US), Ava Multilingual (F, US), Ryan (M, UK), Sonia (F, UK), Libby (F, UK), Alfie (M, UK) |
| Chatterbox |
| Abbey (F, US), Susan (F, US), Alan (M, US), Michael (M, US), Amy (F, UK), Redd (F, UK), Dave (M, UK), Tom (M, UK) |
| Chatterbox HD |
| Susan (F, US), Michael (M, US), Abbey (F, US), Alan (M, US), Redd (F, UK), Tom (M, UK), Dave (M, UK), Amy (F, UK) |
| Chirp 3: HD |
| Aoede (F, US), Achird (M, US), Algenib (M, US), Achernar (F, US), Autonoe (F, UK), Callirrhoe (F, UK), Algieba (M, UK), Alnilam (M, UK) |
| ElevenLabs v3 |
| Laura (F, US), Jessica (F, US), Jarnathan (M, US), Liam (M, US), Elizabeth (F, UK), Shelley (F, UK), Dan (M, UK), Nathaniel (M, UK) |
| Falcon (Beta) |
| Carter (, ), Phoebe (, ), Terrell (, ), Natalie (, ), Theo (, ), Mason (, ), Ruby (, ), Hazel (, ) |
| Fish Speech 1.5 |
| US Female Susan (F, US), US Male Alan (M, US), US Female Abbey (F, US), US Male Michael (M, US), UK Female Redd (F, UK), UK Male Dave (M, UK), UK Male Tom (M, UK), UK Female Amy (F, UK) |
| Flash v2.5 |
| Liam (M, US), Laura (M, US), Jarnathan (M, US), Jessica (M, US), Nathaniel (M, UK), Shelley (F, UK), Dan (M, UK), Elizabeth (F, UK) |
| Gemini 2.5 Flash Lite TTS |
| Achernar (F, US), Achird (M, US), Algenib (M, US), Aoede (F, US), Algieba (M, UK), Alnilam (M, UK), Autonoe (F, UK), Callirrhoe (F, UK) |
| Gemini 2.5 Flash TTS |
| Achernar (F, US), Achird (M, US), Algenib (M, US), Aoede (F, US), Alnilam (M, UK), Autonoe (F, UK), Callirrhoe (F, UK), Algieba (M, UK) |
| Gemini 2.5 Flash TTS (Dec 2025) |
| Algieba (M, UK), Achernar (F, US), Achird (M, US), Algenib (M, US), Aoede (F, US), Alnilam (M, UK), Autonoe (F, UK), Callirrhoe (F, UK) |
| Gemini 2.5 Pro TTS |
| Aoede (F, US), Achird (M, US), Achernar (F, US), Algenib (M, US), Autonoe (F, UK), Alnilam (M, UK), Algieba (M, UK), Callirrhoe (F, UK) |
| Inworld TTS 1 |
| Ashley (F, US), Mark (M, US), Theodore (M, US), Deborah (F, US), Craig (M, UK), Olivia (F, UK), Ronald (M, UK), Wendy (F, UK) |
| Inworld TTS 1 Max |
| Theodore (M, US), Mark (M, US), Deborah (F, US), Ashley (F, US), Wendy (F, UK), Craig (M, UK), Ronald (M, UK), Olivia (F, UK) |
| Inworld TTS 1.5 Max |
| Craig (M, UK), Ronald (M, UK), Wendy (F, UK), Olivia (F, UK), Theodore (M, US), Mark (M, US), Deborah (F, US), Ashley (F, US) |
| Inworld TTS 1.5 Mini |
| Ashley (F, US), Deborah (F, US), Mark (M, US), Olivia (F, UK), Wendy (F, UK), Ronald (M, UK), Craig (M, UK), Theodore (M, US) |
| Journey |
| en-US-Journey-D (M, US), en-US-Journey-F (F, US) |
| Kokoro 82M v1.0 |
| Fenrir (M, US), Bella (F, US), Michael (M, US), Aoede (F, US), George (M, UK), Fable (M, UK), Emma (F, UK), Isabella (F, UK) |
| LMNT |
| daniel (M, US), terrence (M, US), lily (F, US), chloe (F, US), morgan (F, UK) |
| Magpie Multilingual |
| Leo (M, US), Mia (F, US), Aria (F, US), Jason (M, US) |
| Magpie-Multilingual 357M |
| Sofia (F, US), Aria (F, US), Jason (M, US), Leo (M, US), John (M, US) |
| Maya1 |
| Ava (F, US), Noah (M, US), James (M, US), Emma (F, US), Liam (M, UK), Chloe (F, UK), Oliver (M, UK), Sophie (F, UK) |
| MetaVoice v1 |
| Susan (F, US), Michael (M, US), Abbey (F, US), Alan (M, US), Redd (F, UK), Tom (M, UK), Dave (M, UK), Amy (F, UK) |
| Multilingual v2 |
| Jessica (F, US), Laura (F, US), Liam (M, US), Jarnathan (M, US), Nathaniel (M, UK), Dan (M, UK), Shelley (F, UK), Elizabeth (F, UK) |
| Murf Speech Gen 2 |
| Carter (M, US), Phoebe (F, US), Terrell (M, US), Natalie (F, US), Theo (M, UK), Mason (M, UK), Ruby (F, UK), Hazel (F, UK) |
| Neuphonic TTS |
| Emily (F, US), Liz (F, UK), Albert (M, UK), Julia (F, UK), Dave (M, UK), Miles (M, US), Annie (F, US) |
| Neural2 |
| en-US-Neural2-I (M, US), en-US-Neural2-A (M, US), en-US-Neural2-H (F, US), en-US-Neural2-C (F, US), en-GB-Neural2-B (M, UK), en-GB-Neural2-D (M, UK), en-GB-Neural2-C (F, UK), en-GB-Neural2-A (F, UK) |
| Octave 2 |
| FEMALE MEDITATION GUIDE (F, US), MALE PROTAGONIST (M, US), DONOVAN SINCLAIR (M, US), SITCOM GIRL (F, US), NATURE DOCUMENTARY NARRATOR (M, UK), ALICE BENNETT (F, UK), LADY ELIZABETH (F, UK), SAD OLD BRITISH MAN (M, UK) |
| Octave TTS |
| MALE PROTAGONIST (M, US), DONOVAN SINCLAIR (M, US), FEMALE MEDITATION GUIDE (F, US), SITCOM GIRL (F, US), ALICE BENNETT (F, UK), LADY ELIZABETH (F, UK), NATURE DOCUMENTARY NARRATOR (M, UK), SAD OLD BRITISH MAN (M, UK) |
| OpenAudio S1 |
| US Female Susan (F, US), US Male Michael (M, US), US Male Alan (M, US), US Female Abbey (F, US), UK Male Dave (M, UK), UK Female Redd (F, UK), UK Male Tom (M, UK), UK Female Amy (F, UK) |
| OpenVoice v2 |
| Susan (F, US), Michael (M, US), Abbey (F, US), Alan (M, US), Redd (F, UK), Tom (M, UK), Dave (M, UK), Amy (F, UK) |
| Polly Generative |
| Matthew (M, US), Ruth (F, US), Danielle (F, US), Stephen (M, US) |
| Polly Long-Form |
| Gregory (M, US), Danielle (F, US), Ruth (F, US) |
| Polly Neural |
| Joey (M, US), Gregory (M, US), Joanna (F, US), Danielle (F, US), Brian (M, UK), Brian (M, UK), Amy (F, UK), Emma (F, UK) |
| Polly Standard |
| Joey (M, US), Joanna (F, US), Brian (M, UK), Amy (F, UK) |
| Qwen3 TTS Flash |
| Cherry (F, US), Ryan (M, US), Jennifer (F, US), Ethan (M, US) |
| Simba |
| Patricia (F, US), Robert (M, US), Douglas (M, US), Christina (F, US), Austin (M, UK), Derek (M, UK), Beverly (F, UK), carol (F, UK) |
| Sonic 3 |
| Kiefer (M, US), Brandon (M, US), Tessa (F, US), Linda (F, US), Fiona - Witty Woman (F, UK), Redd (F, UK), Tom (M, UK), Benedict (M, UK) |
| Sonic English (Oct '24) |
| Nonfiction Man (M, US), Newsman (M, US), Helpful Woman (F, US), Classy British Man (M, UK), Southern Woman (F, US), Polite Man (M, UK), British Lady (F, UK), British Narration Lady (F, UK) |
| Speech 2.6 HD |
| Captivating Storyteller (M, US), Insightful Speaker (M, US), Captivating Female (F, US), Upbeat Woman (F, US), Graceful Lady (F, UK), Patient Man (M, UK), Expressive Narrator (M, UK), Compelling Lady (F, UK) |
| Speech 2.6 Turbo |
| Captivating Storyteller (M, US), Insightful Speaker (M, US), Captivating Female (F, US), Upbeat Woman (F, US), Patient Man (M, UK), Expressive Narrator (M, UK), Compelling Lady (F, UK), Graceful Lady (F, UK) |
| Speech-02-HD |
| English Sweet Female (F, US), English Lively Male (M, US), English Powerful Female (F, UK), English_Magnetic_Male_2 (M, UK) |
| Speech-02-Turbo |
| English Lively Male (M, US), English Sweet Female (F, US), English Magnetic Male (M, UK), English Powerful Female (F, UK) |
| Standard |
| en-US-Standard-F (F, US), en-US-Standard-I (M, US), en-US-Standard-A (M, US), en-US-Standard-C (F, US), en-GB-Standard-A (F, UK), en-GB-Standard-D (M, UK), en-GB-Standard-C (F, UK), en-GB-Standard-B (M, UK) |
| Step Audio EditX |
| Amy (F, UK), Alan (M, US), Abbey (F, US), Michael (M, US), Susan (F, US), Dave (M, UK), Tom (M, UK), Redd (F, UK) |
| Step TTS 2 |
| Lively Girl (F, US), Vibrant Youth (M, US) |
| Step TTS Mini |
| Susan (F, US), Abbey (F, US), Michael (M, US), Alan (M, US), Tom (M, UK), Dave (M, UK), Redd (F, UK), Amy (F, UK) |
| Studio |
| en-US-Studio-Q (M, US), en-US-Studio-O (F, US), en-GB-Studio-C (F, UK), en-GB-Studio-B (M, UK) |
| StyleTTS 2 |
| Susan (F, US), Michael (M, US), Abbey (F, US), Alan (M, US), Redd (F, UK), Tom (M, UK), Dave (M, UK), Amy (F, UK) |
| T2A-01-HD |
| Mature Boss (F, US), English Steady Mentor (F, US), Gentle-voiced_man (M, US), English Whimsical Girl (F, US), Decent Young Man (M, UK), Wise Scholar (M, UK), Female Narrator (F, UK), Wise Lady (F, UK) |
| T2A-01-Turbo |
| English Steady Mentor (M, US), English Whimsical Girl (F, US), English Gentle Voiced Man (M, US), Boss Lady (F, US), Wise Lady (F, UK), English Wise Scholar (M, UK), Decent Young Man (M, UK), Anime Character (F, UK) |
| TTS-1 |
| echo (M, US), alloy (F, US), nova (F, US), shimmer (F, US), onyx (M, US), fable (M, UK) |
| TTS-1 HD |
| nova (F, US), alloy (F, US), onyx (M, US), shimmer (F, US), echo (M, US), fable (M, UK) |
| Turbo v2.5 |
| Laura (F, US), Jessica (F, US), Jarnathan (M, US), Liam (M, US), Elizabeth (F, UK), Shelley (F, UK), Dan (M, UK), Nathaniel (M, UK) |
| VibeVoice 1.5B |
| Maya (F, US), Alice (F, US), Carter (M, US), Frank (M, US) |
| VibeVoice 7B |
| Alice (F, US), Maya (F, US), Carter (M, US), Frank (M, US) |
| WaveNet |
| en-US-Wavenet-I (M, US), en-US-Wavenet-B (M, US), en-US-Wavenet-C (F, US), en-US-Wavenet-F (F, US), en-GB-Wavenet-B (M, UK), en-GB-Wavenet-D (M, UK), en-GB-Wavenet-C (F, UK), en-GB-Wavenet-A (F, UK) |
| XTTS v2 |
| Susan (F, US), Michael (M, US), Abbey (F, US), Alan (M, US), Redd (F, UK), Tom (M, UK), Dave (M, UK), Amy (F, UK) |
| Zonos-v0.1 |
| Abbey (F, US), Susan (F, US), Alan (M, US), Michael (M, US), Dave (M, UK), Tom (M, UK), Amy (F, UK), Redd (F, UK) |
Rime
async
Microsoft Azure
Resemble AI
Google
ElevenLabs
Murf AI
Fish Audio
Inworld
Kokoro
LMNT
NVIDIA
Maya Research
MetaVoice
Neuphonic
Hume AI
OpenVoice
Amazon
Alibaba
Speechify
Cartesia
MiniMax
StepFun
StyleTTS 
OpenAI
Coqui
ZyphraModel and Provider Inclusion Criteria
Our objective is to analyze and compare popular and high-performing Text to Speech models and providers to support end-users in choosing which to use. As such, we apply an 'industry significance' and competitive performance test to evaluate the inclusion of new models and providers. We are in the process of refining these criteria and welcome any feedback and suggestions. To suggest models or providers, please contact us via the contact page.
Benchmarking is conducted 4 times daily at random times each day. For each benchmarking evlauation we select a single random voice for each model. A unique prompt of ~500 characters is used for each generation.
Statement of Independence
Benchmarking is conducted with strict independence and objectivity. No compensation is received from any providers for listing or favorable outcomes on Artificial Analysis.