Gradium TTS: you no longer have to choose between latency and accuracy

A voice agent giving you a tracking number has to be fast and correct: that's table stakes. Speed and accuracy usually pull against each other. Our TTS model gives you both: it reads your hard cases correctly, in real time.

A new Gradium TTS model is available today, and it is now the default. It passes 81.0% of a 500-sentence hard-case evaluation across five languages, ahead of every model we tested it against, and its time to first audio (TTFA) P50 is 216 ms. It reads the cases that break voice agents in production, phone numbers, email addresses, IBANs, with no pre-processing or text normalization on your side. We opened the public beta on July 30 for exactly these cases and asked for failures. This release is what came out of that, and the evaluation set we built from those reports is open-sourced on Hugging Face.

What this release fixes: hallucinations, text normalization and latency

Reliability:

• No hallucinations. The model reads what it is given, including the parts that are not prose.
• Speech intelligence. Nobody should have to spell out $1.5M as "one point five million dollars" in a voice agent prompt, next to the personality and the tool calls. Those workarounds are tied to one model at one version and break on the next release. We are building models intelligent enough to semantically choose how to pronounce a sequence of characters.
• Straightforward testing. Some TTS labs quietly rewrite your text with an LLM before synthesizing it in their studio, so the demo pronounces things the API never will. We do not do that. No rewriting, no hidden normalization, same input path as production, because the point of testing in the studio is to hear exactly what your users will hear.

Lower latency:

• On Coval, TTFA P50 is 216 ms, 170 ms faster than the model this replaces. For a live call, what matters as much as the median is how far a bad turn strays from it. Our p75 - p25 spread is 30 ms across 480 runs, the tightest of the five models, so the turn a caller actually waits on lands close to the median.
• Cartesia Sonic 3.6 sits at 454 ms median with a 165 ms spread, 36% of its own median. A caller does not experience your median.

Methodology, measurement point, regions and run counts are published with the benchmark at benchmarks.coval.ai/tts, and we'll share more in another post.

Robust on the hard cases: spelling, numbers, emails, dates and reference codes

Structured entities are where a lot of TTS models break, and they carry most of what an AI agent call is about: the confirmation number, the amount, the appointment slot, the address the caller has to write down. A digit dropped from a callback number ends the call as surely as a crash.

To measure this properly we built a dataset drawn from the hardest sentences representative of our user base. We are publishing the dataset for reference on Hugging Face. 100 sentences per language in English, German, French, Spanish and Portuguese, 500 in total.

Gradium TTS passes 81.0% of the set, ahead of Cartesia Sonic 3.6 at 75.1%, ElevenLabs v3 Conversational at 65.4%, Fish Audio S2.1 Pro at 49.5% and Inworld TTS 1.5 Max at 46.5%. That is the overall pass rate, pooled across the ten published criteria and averaged over the five languages, each weighted equally. A sentence passes only if an independent native-speaker rater hears every element pronounced correctly and completely. One dropped digit fails the sentence, because one dropped digit fails the call.

Every pass/fail test follows the same controls: normalize for loudness, randomize the order, cap raters at 40 comparisons and 2 sessions in a row, and require a 5-minute break to limit fatigue.

Across all ten criteria, our model offers the best balance on the types of hard cases a voice agent meets. We'll continue to improve, as our models are always training.

If you are hitting harder cases in production, sentences this set does not cover and where our model fails, we want them: send the input text and the language on Discord. They go into the next evaluation run and we'll grant 1M credits for all complete submissions.

Hear the failures

The pass rate says how often a model gets it right. Here is what that sounds like.

What comes next for Gradium TTS

We are not finished on latency, and the next step is a larger one than the gains in this release. Continuous improvements on accuracy, naturalness and multilingual coverage are the other tracks.

Iteration stays fast and driven by what you report. Thanks to all the users who have given feedback, and please continue to submit ideas on our public feedback form.

Get started

The model is available through the API and in Studio. It's now the default, so you have nothing to do to benefit from the gains. Your existing voices, including custom voices, work as-is. And to get started with Gradium, here is the Python SDK:

import gradium

client = gradium.client.GradiumClient(api_key="your-api-key")

result = await client.tts(
    setup={
        "voice_id": "Bla6SbVMczYnOhfK",
        "output_format": "wav",
    },
    text="Reference RF24, surname spelled F-O-R-S-Y-T-H.",
)

with open("output.wav", "wb") as f:
    f.write(result.raw_data)

Running on Pipecat or LiveKit? Both are supported directly. Check-out our integrations guides.

If you use a coding agent, paste this prompt to wire it up (you need a Gradium API key first):

Install the Gradium Python SDK and write a script that synthesizes speech
with model_name "default", voice_id "Bla6SbVMczYnOhfK", WebSocket endpoint
wss://api.gradium.ai/api/speech/tts, API key from the GRADIUM_API_KEY env
var. Use the docs at
https://docs.gradium.ai/api-reference/endpoint/tts-websocket#setup.