How to Add Text-to-Speech to Your App, Website, or Voice Agent

Adding TTS to a product is a different problem depending on what you are building. A website that reads content aloud, a mobile app with offline narration, and a real-time voice agent all need TTS, but in different architectures, with different latency requirements, and integrated in different ways.

This guide covers the three most common integration paths: direct WebSocket streaming for real-time applications, orchestration framework integration (LiveKit and Pipecat) for voice agents, and on-device deployment for offline or high-volume consumer apps.

Before you start: what integration path do you need?

The right integration depends on one question: does the user wait for audio in real time?

Real-time voice agent: the user speaks, the agent responds, and the audio must start within 200 to 300 ms. Use WebSocket streaming TTS, either directly or via an orchestration framework like LiveKit or Pipecat. The Gradium TTS API is WebSocket-native and delivers 158 ms TTFA P50 (Coval benchmark, May 13, 2026).

Website or app with non-interactive narration: the audio is generated in response to a user action but does not need to feel conversational. WebSocket streaming still improves perceived responsiveness, but the latency requirement is looser. A direct API call works.

On-device or offline: text cannot leave the device, or per-request cloud pricing does not scale for your user base. Use Gradium Phonon, the on-device TTS model that runs on a single CPU core with no network dependency. Currently in private beta at gradium.ai/on-device-tts.

Path 1: direct WebSocket integration

What you need

A Gradium account and API key from Gradium Studio. The free plan provides 45,000 credits per month with no credit card required. Documentation is at docs.gradium.ai.

How the WebSocket lifecycle works

Gradium's TTS API uses a WebSocket connection with a specific message sequence. Every message sent or received is a JSON-serialized object with a type field.

As Olivier Teboul, CTO and co-founder of Gradium, explains in the TTS WebSocket API walkthrough: "the very first message must be the setup message. In it, we define the voice and the output format. If everything is valid, the server will confirm that the stream is initialized and ready to receive text messages."

The full sequence:

1. Open a WebSocket connection to the TTS endpoint, passing your API key in the request header
2. Send a setup message defining the voice ID, output format (PCM, WAV), and optional parameters such as json_config for normalization rules and pronunciation_id for custom pronunciation dictionaries
3. Receive a ready message confirming the stream is initialized
4. Send one or more text messages, each containing the text to synthesize
5. Receive audio chunks encoded as base64 strings, alongside word-level timestamps
6. Send an end-of-stream message when you have finished sending text
7. Receive the server's end-of-stream message and the connection closes

Gradium's TTS outputs 16-bit PCM audio at 48 kHz by default, with 16 kHz and 24 kHz available as configurable options. Audio starts streaming before synthesis of the full text is complete, which is what enables sub-200 ms TTFA.

Configuring voice and output

The setup message is where voice and quality settings are defined. The key parameters:

• voice_id: a voice ID from Gradium's catalogue or a cloned voice ID
• model_name: defaults to the current default model if omitted
• output_format: output audio format, typically PCM for voice agents (raw frames, no WAV header)
• json_config: optional, enables normalization rules for dates, phone numbers, emails, and other structured content
• pronunciation_id: optional, attaches a pronunciation dictionary for domain-specific terms
• Codebook depth: controls the quality/latency tradeoff

Gradium documents three codebook configurations for different deployment contexts:

For further control over structured text handling, see How to Handle TTS Edge Cases with Text Normalization in Gradium and How to Use Pronunciation Dictionaries in Gradium TTS.

Reducing per-turn overhead with multiplexing

In multi-turn applications where the same user triggers multiple TTS requests, WebSocket multiplexing reuses a single persistent connection across all of them. In a 10-turn conversation with 50 ms connection overhead per turn, multiplexing saves approximately 450 ms of accumulated latency compared to HTTP-per-request architectures. Gradium supports multiplexing, reducing effective TTFA to 214 ms P50 in standard configuration. The full implementation is in How to Multiplex TTS Requests Over One WebSocket Connection and the multiplexing guide.

Path 2: voice agent integration with LiveKit

LiveKit is the recommended integration path for real-time voice agents. Gradium ships as a native LiveKit plugin, and the full agent (STT, LLM, TTS, turn-taking) can be assembled in approximately 100 lines of Python.

Installation

pip install "livekit-agents[gradium]~=1.3"

This single command installs both Gradium STT and TTS as ready-to-use LiveKit plugins.

Environment setup

LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your-livekit-key
LIVEKIT_API_SECRET=your-livekit-secret
GRADIUM_API_KEY=your-gradium-key

Agent session

The AgentSession is the core of the LiveKit agent. It declares which models handle STT, LLM, and TTS, and manages the full real-time loop:

session = AgentSession(
    stt=gradium.STT(vad_threshold=0.6, vad_bucket=1),
    llm=inference.LLM(model="openai/gpt-4.1-mini"),
    tts=gradium.TTS(),
    allow_interruptions=True,
    min_interruption_words=0,
    preemptive_generation=True,
)

Key parameters:

• vad_threshold and vad_bucket control Gradium's semantic VAD sensitivity. Semantic VAD determines when the user has finished a complete thought, not just stopped making sound, which removes the awkward pauses that rule-based silence timers introduce.
• allow_interruptions=True lets users speak over the agent at any point.
• preemptive_generation=True starts LLM response generation before the user finishes speaking, reducing perceived response latency.

Agent class and function tools

class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a voice assistant for..."
        )

    @function_tool
    async def lookup_profile(self, name: str) -> str:
        """Look up a user's saved profile."""
        ...

The agent receives system instructions and Python function tools it can call during conversation.

Testing and deployment

# Download required models
uv run python src/agent.py download-files

# Test in terminal
uv run python src/agent.py console

# Run connected to LiveKit Cloud
uv run python src/agent.py dev

# Deploy to production
lk cloud deploy

Ensure GRADIUM_API_KEY is added as a secret in LiveKit Cloud before deploying. Without it, the agent cannot process speech or generate audio.

The full walkthrough, from account setup through production deployment, is in How to Build a Voice AI Agent with Gradium and LiveKit. For a Pipecat-based setup, see How to Build an Audiobook Agent with Gradium and Pipecat.

Path 3: voice agent integration with Pipecat

Pipecat is an open-source Python framework for voice and multimodal agents. Gradium is available as a Pipecat service:

uv add "pipecat-ai[gradium]"

from pipecat.services.gradium import GradiumTTSService

tts = GradiumTTSService(
    api_key=os.getenv("GRADIUM_API_KEY"),
    settings=GradiumTTSService.Settings(
        voice="your-voice-id",
        model="default",
    ),
)

Pipecat's GradiumTTSService supports word-level timestamps, runtime voice switching via TTSUpdateSettingsFrame (which automatically reconnects the WebSocket with the new voice applied), and the standard Pipecat connection event handlers (on_connected, on_disconnected, on_connection_error).

The full guide for the Pipecat integration is at Gradium and Pipecat: Native TTS Integration for Voice Agents.

Path 4: fast prototype with Gradbot

For a quick prototype or hackathon build, Gradbot is Gradium's open-source framework for assembling a working voice agent in under 50 lines of Python with any OpenAI-compatible LLM. It handles VAD, turn-taking, fillers, and interruptions automatically. The developer defines the agent's instructions and any tools it should call.

Gradbot is designed for prototyping and first MVPs, not for production at scale. For production, use LiveKit or Pipecat as the orchestration layer.

Repository and documentation: gradium.ai/gradbot.

Summary: which integration path should you use?

Glossary

WebSocket TTS

A transport architecture for TTS that maintains a persistent bidirectional connection between the client and the TTS server. Delivers audio incrementally as it is generated. Required for sub-200 ms TTFA. Gradium's TTS API is WebSocket-native.

AgentSession (LiveKit)

The core abstraction in LiveKit's Agents framework. Accepts STT, LLM, and TTS model declarations and manages the full real-time audio loop, including VAD, interruption handling, and audio transport.

GradiumTTSService (Pipecat)

The Pipecat service class for Gradium TTS. Installable via pipecat-ai[gradium]. Supports runtime voice switching, word-level timestamps, and standard Pipecat connection events.

Semantic VAD

Voice Activity Detection that uses the linguistic meaning of an utterance to determine when a speaker has finished their turn, rather than relying on silence duration alone. Native to Gradium's STT and configurable in LiveKit via vad_threshold and vad_bucket.

WebSocket multiplexing

A technique that reuses a single persistent WebSocket connection across multiple concurrent TTS sessions, eliminating per-turn reconnection overhead. Gradium supports multiplexing, reducing effective TTFA to 214 ms P50 in standard configuration.

json_config

A parameter in Gradium's TTS WebSocket setup message that enables text normalization rules. Used to control how structured content (phone numbers, dates, emails, URLs, alphanumeric codes) is synthesized. Pronunciation dictionaries are attached separately via the pronunciation_id setup parameter.