Open-Sourcing Phonon: A Suite of Frontier On-Device TTS Models

Today, we're open-sourcing Phonon, now a family of 31m and 40m parameter text-to-speech (TTS) models that set a new standard for naturalness and accuracy on-device, making natural, high-quality speech possible even on the most resource-constrained hardware.

Phonon pushes the frontier of speech quality at this scale. Our English models deliver best-in-class naturalness and accuracy, while our multilingual models mark an even bigger leap forward, dramatically advancing the state of the art across five languages.

And they're exceptionally fast. Phonon 31m starts speaking in just 7 ms on a laptop CPU and is available under Apache 2.0. Phonon 40m comes in both language-specific variants and a single multilingual model supporting English, French, German, Spanish, and Portuguese with the same weights. All models are optimized for more than 10 major hardware platforms: CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU), and NPU (Apple Core ML, Snapdragon).

Type a sentence in the box below. The voice you hear is generated inside this browser tab, with no server involved.

A new wave of language models recently appeared starting with Kyutai’s effort on Pocket TTS. These models are small enough to run on a laptop or a phone, and still provide high-quality speech in real-time [4]. Until now, on-device voice still meant compromises: limited language coverage, models too large for the memory budget, or quality too low to ship. Phonon is built for local inference first: the text stays on the device, the voice works without a network, and usage is unlimited at no extra cost. Every app that speaks brings voice one step closer to being the default interface between humans and machines.

Phonon does three things on device:

• Speaks with any voice: 100+ built-in, or any voice you clone or design with Gradium
• Speaks any text, including numbers, dates and codes, in five languages
• Runs anywhere: phones, laptops, single-board computers and web browsers

We release these models as open weight to empower the community to create new voice-powered applications. Here are some demos built by our friends at Autonomous, Modal and Xybrid AI.

What we release today: a suite of open-weight Phonon models

Phonon is the open release of the model we announced in April [1]. Since then it has been in private beta, and we have built on it with our partners and early users [2][3]. Their feedback shaped this release: smaller sizes, more languages and support for any custom voice, e.g. via voice design.

It comes in two sizes, so you can trade footprint for quality depending on where it runs. We release the English Phonon 31m under an Apache 2.0 license so developers can build new applications on it freely, from prototypes to commercial products. Phonon 40m is the one to pick when expressivity and accuracy on hard text matter most: it gets closer to larger models and comes in 5 languages. Phonon 40m multilingual covers five languages in a single model, such that a single set of weights can serve more users in your apps. Each use case comes with specific memory constraints, language requirements or fine-tuning needs: reach out to us here for commercial usage and custom models.

All models read numbers, dates and other structured text correctly through text normalization, and let you set the speaking speed and normalize loudness.

Benchmarks: naturalness, accuracy and inference performance

Naturalness (ELO)

Developers judge a TTS model first by how it sounds. We had independent listeners compare Phonon with Kokoro, Magpie, NeuTTS and Pocket TTS in blind pairwise tests, in each language (1,189 raters, 17,270 votes).

Accuracy (WER)

TTS models are known to struggle with complex structured data, typically phone numbers, email addresses and postal codes. Our models use text normalization to maximize performance on this type of data. We built a test set of 500 sentences per language from hard cases developers sent us, the ones their current models get wrong. On this set, Phonon 31m and 40m make fewer errors than any other model of their size. The smallest model trades some accuracy for size; for tight memory budgets and specific kinds of text, a custom model trained on your use case gets you both.

The test set is public on Hugging Face: gradium/tts-hard-sentences-202610.

Inference performance: supported devices

Phonon ships with its own inference stack and q8 GGUF weights, with backends for CPU, Metal, CUDA, Vulkan, WebGPU and Apple Core ML. On a MacBook Air M5 CPU, Phonon starts speaking in 7 ms; Kokoro takes 517 ms. On a Raspberry Pi 5, Phonon 31m starts speaking in 38 ms and runs 3.7 times faster than real time; Kokoro needs 7 seconds to start and runs below real time.

Phonon runs on Arm and Snapdragon chips inside today’s phones and laptops, and both launch partners helped us make it fast:

Arm is the foundational compute platform for mobile AI, powering 99% of the world's smartphones, and is helping enable a new generation of AI-powered PCs. KleidiAI is Arm’s library of optimised building blocks that makes AI models run faster on Arm processors. Phonon integrates KleidiAI, so developers running Phonon on Arm hardware get faster inference out of the box. Phonon runs on Arm CPUs with KleidiAI acceleration: 28 ms to first audio and a real-time factor of 0.066, measured by our partners at Arm.

Arm. Arm CPUs power nearly every smartphone and every Apple Silicon Mac. Phonon integrates KleidiAI, Arm’s library of optimized building blocks for AI, so it runs faster on Arm hardware out of the box: 28 ms to first audio and 15x faster than real time, as measured by our partners at Arm.

Qualcomm. With our partners at Qualcomm, we’re bringing Phonon to Snapdragon. On the Snapdragon 8 Elite Gen 5, Phonon starts speaking in 7.5 ms and runs 24x faster than real time.

Custom voices for every app

Most on-device models ship with a limited set of pre-packaged voices. But every product needs its own: character voices for a game, an angry pirate for a themed app, an encouraging coach for language learning. Voice cloning helps, but is hard to scale: every voice needs a speaker, their consent and a recording.

Phonon works with both instant voice cloning and voice design. Clone a voice from a short consented recording, or describe a new one in plain text, in Gradium Studio or through the API. Download the voice file, load it in Phonon, and it runs fully on device. Creating voices is free and unlimited, on top of the 102 built-in ones, including 12 English character voices made with voice design, so every user of your app can have their own.

How Phonon works

Phonon builds on our earlier research at Kyutai on Continuous Audio Language Models (arXiv), which led to the development of Pocket TTS and demonstrated real-time, high-quality speech synthesis on consumer CPUs [4], with latest developments that push miniaturization even further (Orsini, 2026). At Gradium, we are extending this research towards accurate, multilingual, and controllable on-device speech synthesis for real-world applications.

Phonon is based on a causal transformer backbone. The backbone takes as prefix the text embeddings concatenated with the speaker embeddings. The text embeddings come from a SentencePiece tokenizer with a 4,000-token vocabulary, so the model reads raw text with no phoneme conversion, which makes it easy to extend Phonon to new languages.

The backbone is then used in an auto-regressive way, generating a latent vector for each 80 ms frame. A flow-matching head turns this vector, together with random noise, into continuous audio embeddings in a single step. These audio embeddings are converted to 80 ms of 24 kHz PCM audio data via a smaller variant of the Mimi decoder. An end-of-speech (EOS) head also operates on the latent vector to decide when to stop the generation.

The different Phonon model sizes correspond to a different number of layers and hidden size used in the backbone. All the models are distilled from a larger Phonon teacher. The same flow-matching head and Mimi decoder are shared by all the Phonon models.

All Phonon variants were trained on a cluster of NVIDIA H100 GPUs, using NCCL for distributed training and CUDA Graphs to speed up research experiments on very small models.

Custom models for your product

At this size, a model built around your constraints beats a generic one. So we also train custom Phonon models with design partners: smaller ones for tight memory budgets, other languages including Korean, Arabic and Japanese, or models fine-tuned on your own vocabulary.

Automotive is a good example. A voice in the car has to run on embedded hardware with limited memory, respond instantly so the driver isn’t kept waiting, keep working in tunnels and areas with no coverage, and handle the languages of every market the car ships to. FORVIA and Appning by FORVIA are design partners using Phonon in joint R&D in an automotive context.

Get started

Download the weights from Hugging Face: Phonon 31m and Phonon 40m. The inference code is on GitHub at github.com/gradium-ai/xn-ptts, and the README covers installation, a quick-start and the integrations for Rust, Python (with a CLI), Swift and JavaScript/WASM. The repo also includes a WebSocket server and an OpenAI-compatible server. Point your coding agent at it and start building.

Using a coding agent? Paste this prompt:

Phonon is also available in the tools and libraries you already use:

• MLX-Audio: run Phonon natively on Apple Silicon from day 0.
• Transformers: load Phonon from the Hugging Face Hub.
• LM Studio: Phonon powers Read aloud in Bionic by LM Studio.
• Ollama: Phonon is integrated in Ollama.

Need a smaller model, another language or a model fine-tuned for your product? Reach out here.

Gradium Phonon is our first open release. Tell us what you build.

References

[1] Gradium Phonon: on-device TTS for consumer apps, NPCs and offline products, 3 April 2026

[2] Evaluating Phonon: how we made the best TTS model for edge devices, 9 April 2026

[3] Phonon update: 1.00% WER on Seed-TTS, 26 May 2026

[4] Kyutai, Pocket TTS blog post (kyutai.org, January 2026)