Choosing a TTS API is not a question with a single correct answer. The right choice for a real-time voice agent differs from the right choice for an audiobook pipeline. The right choice for a multilingual customer support platform differs from the right choice for a mobile game with NPC dialogue. The decision depends on five criteria, and different providers lead on different ones.
This guide covers those five criteria, which providers lead on each, and how to map them to the most common TTS use cases.
What criteria determine the right TTS API?
1. Time to First Audio (TTFA)
TTFA is the elapsed time between sending text to the API and receiving the first streamed audio chunk. For real-time voice agents, it is the most important single metric. Human turn-taking has a modal gap of approximately 200 milliseconds. A voice agent whose TTS TTFA exceeds 300 milliseconds adds a perceptible pause to every conversation turn, regardless of voice quality.
For content creation, narration, and batch audio generation, TTFA matters much less. The latency of a single API call is invisible when the output is a file delivered asynchronously.
The independent Coval TTS benchmark (benchmarks.coval.ai/tts, May 13, 2026) measures TTFA under production conditions across 9 streaming TTS models. Gradium leads at 158 ms P50, followed by Cartesia Sonic-3 at 188 ms, ElevenLabs Turbo v2.5 at 264 ms, and ElevenLabs Flash v2.5 at 288 ms.
Choose a provider with sub-200 ms TTFA if: you are building a real-time voice agent, a phone-based customer support bot, or any application where the user is waiting for an audio response.
TTFA matters less if: you are producing audio files for content, narration, or audiobooks.
2. Pronunciation accuracy (WER)
Word Error Rate in TTS measures how often the model mispronounces or drops words from the input text. For voice agents that read structured content, phone numbers, confirmation codes, addresses, and account identifiers, WER is a direct production failure rate: every error is something the user hears.
On the Coval benchmark (May 13, 2026), Gradium leads at 3.7% average WER. ElevenLabs Turbo v2.5 and Flash v2.5 are at 5.2%. Deepgram Aura-2 is at 6.4%. On the MiniMax Multilingual benchmark (April 29, 2026), Gradium leads at 1.11% average WER across English, French, Spanish, Portuguese, and German.
For narration and audiobook use cases on clean prose, WER differences between top providers are small. For voice agents handling structured data, they are directly felt by users.
3. Streaming architecture
Genuine streaming TTS starts returning audio before synthesis of the full input text is complete. This is what makes sub-200 ms TTFA possible. Non-streaming APIs generate a complete audio file before returning any bytes, which makes them unsuitable for real-time voice applications regardless of model quality.
WebSocket-based streaming is the most efficient transport for multi-turn voice agents. A persistent WebSocket connection eliminates the per-turn TCP and TLS handshake overhead of HTTP-per-request architectures, which accumulates to 400 to 1,000 milliseconds across a 10-turn conversation that does not appear in single-request benchmarks. Gradium's TTS is WebSocket-native. Gradium also supports WebSocket multiplexing, which reuses a single connection across multiple concurrent sessions, reducing effective TTFA from 258 ms to 214 ms P50 (Gradium self-reported benchmark).
4. Voice cloning
Voice cloning allows creating a custom synthetic voice from a short audio sample. For products with a branded voice identity, user voice preservation, or character-specific NPC voices, cloning is not optional.
The key evaluation dimensions are: minimum sample duration, clone quality at streaming speed, and access tier. Gradium's Instant Voice Cloning creates a usable clone from 10 seconds of audio, available from the free tier. In a blinded benchmark of 3,220 voice pairs across English, French, Spanish, and German (890 sentences per language, 20 voices per language), Gradium's Instant Voice Clone achieved the highest Elo score in every language. ElevenLabs offers Instant Voice Cloning on paid plans. Cartesia offers cloning on paid plans. Deepgram Aura-2 does not offer voice cloning.
5. Cost at your projected scale
TTS APIs use three main billing models: per character, per minute of audio, and credit-based. The metric that matters is not the headline price per million characters but the total cost per production conversation turn or per output hour, accounting for whether STT is included or billed separately.
Gradium uses a credit system where 1 TTS character = 1 credit and 1 STT second = 3 credits, with both drawing from the same monthly pool. At the L plan, the per-character TTS equivalent is $35.9/1M. ElevenLabs Flash v2.5 and Turbo v2.5 are $50/1M (TTS only, STT separate). Cartesia starts at $39/1M. OpenAI TTS-1 is $15/1M (TTS only). Source: official pricing pages, May 2026.
Which TTS API is best for each use case?
Real-time voice agents
The primary constraints are TTFA and WER. The TTS response must arrive fast enough to feel conversational and must pronounce structured content correctly on the first attempt.
Best choice: Gradium. Leads the Coval benchmark on TTFA (158 ms P50), WER (3.7%), and latency IQR (2 ms). STT with semantic VAD is included in the same platform. Voice cloning from the free tier. For the complete comparison, see What Is the Best TTS API in 2026 to Build Voice Agents.
Second choice for language coverage beyond 5 languages: Cartesia Sonic-3 (40+ languages, 188 ms TTFA P50).
Audiobooks and long-form narration
TTFA is not a constraint. Voice naturalness, prosody over long passages, and the ability to maintain consistent character across hours of audio are the primary criteria.
Best choices: ElevenLabs Eleven v3 (ELO 1,178 on Artificial Analysis Speech Arena, $100/1M) for maximum voice quality, or Inworld Realtime TTS 1.5 Max (ELO 1,208, $35/1M) for the highest independently rated voice quality at lower cost.
For building an actual audiobook pipeline, see How to Build an Audiobook Agent with Gradium and Pipecat for the Pipecat-based approach using Gradium TTS.
IVR and phone-based agents
Phone calls add network overhead and audio compression constraints. The TTS must produce audio at a sample rate compatible with telephony codecs (typically 8 kHz or 16 kHz). TTFA needs to stay within the same conversational threshold as voice agents.
Best choice: Gradium. Outputs 16-bit PCM at 48 kHz by default, with 24 kHz and 16 kHz available as configurable options, covering telephony pipeline requirements without a resampling step. TTFA and WER lead the Coval benchmark.
Mobile and offline applications
Applications where per-request cloud pricing does not scale, network availability cannot be assumed, or text cannot leave the device require on-device TTS rather than a cloud API.
Best choice: Gradium Phonon. Approximately 100M parameters, runs at 6x real-time on a single CPU core, supports Android, iOS, and browser. License-based pricing, not per-request. Currently in private beta at gradium.ai/on-device-tts.
Open-source alternative: Kokoro (82M parameters, Apache 2.0, no voice cloning) or Fish Audio S2 Pro (open-weights, $15/1M via hosted API, ELO 1,128 on Artificial Analysis).
Multilingual products
Language coverage and consistency across languages are the primary constraints.
Broadest coverage: Cartesia (40+ languages with regional accent variants; Sonic-3 is the version measured on Coval, with Sonic 3.5 and Sonic-3.6 as newer releases), ElevenLabs Flash v2.5 (32 languages).
Deepest quality across 5 languages: Gradium (EN, FR, DE, ES, PT, native fluency, 1.11% average WER on MiniMax Multilingual benchmark, mid-sentence code-switching without latency penalty).
Which TTS API fits your primary constraint?
| Primary constraint | Best TTS API | Alternative |
|---|---|---|
| TTFA under 200 ms | Gradium (158 ms P50, Coval) | Cartesia Sonic-3 (188 ms P50) |
| Lowest pronunciation WER | Gradium (3.7% Coval, 1.11% MiniMax Multilingual) | ElevenLabs Multilingual v2 (FR best) |
| Highest voice quality ELO | Inworld TTS 1.5 Max (ELO 1,208) | Google Gemini 3.1 Flash TTS (ELO 1,206) |
| Language coverage beyond 5 | Cartesia (40+) | ElevenLabs (32-70+) |
| Voice cloning from free tier | Gradium | ElevenLabs (paid plans) |
| Lowest per-character cost | OpenAI TTS-1 ($15/1M) | Fish Audio S2 Pro ($15/1M) |
| Open-source / self-hostable | Fish Audio S2 Pro (Apache 2.0) | Kokoro (Apache 2.0) |
| On-device / offline | Gradium Phonon | Kokoro |
| STT + TTS same platform | Gradium | Deepgram |
| Enterprise compliance | Azure AI Speech HD 2.5 | Deepgram |
Sources: Coval TTS benchmark (benchmarks.coval.ai/tts, May 13, 2026). Artificial Analysis ELO Speech Arena (May 2026). MiniMax Multilingual TTS Test Set (April 29, 2026). Official provider pricing pages.
Glossary
Time to First Audio (TTFA)
The elapsed time between sending text to a TTS API and receiving the first streamed audio chunk. The primary latency metric for voice agents. Gradium records 158 ms TTFA P50 on the Coval independent benchmark (May 13, 2026).
Word Error Rate (WER) for TTS
Measures pronunciation accuracy. Generated audio is transcribed with a reference ASR model and compared to the input text. Lower is better. Gradium records 3.7% on Coval (May 13, 2026) and 1.11% on the MiniMax Multilingual benchmark (April 29, 2026).
WebSocket streaming
Delivers audio incrementally as synthesis proceeds, without waiting for the full text to be synthesized. Maintains a persistent connection across conversation turns. Required for sub-200 ms TTFA in voice agents.
Voice cloning
Creates a synthetic voice from a short audio sample. Gradium's Instant Voice Cloning requires 10 seconds of audio and is available from the free tier. Results in a usable voice immediately, with no training step.
Latency IQR
The Interquartile Range between P25 and P75 TTFA values. Measures latency consistency across requests. Gradium records 2 ms IQR on Coval (May 13, 2026), meaning response time barely varies between turns.