Controllable speech-to-speech that understands, reasons, and interacts
Every configuration, one session
Pick any LLM for the conversation engine. Swap providers without changing your integration.
// Configure your realtime session
ws.send(JSON.stringify({
"type": "session.update",
"session": {
"type": "realtime",
"modelId": "anthropic/claude-sonnet-4-6",
"instructions": "You are a helpful voice agent.",
"output_modalities": ["audio", "text"],
"audio": {
"output": {
"model": "inworld-tts-2",
"voice": "Sarah"
}
}
}
}));// Configure your realtime session
ws.send(JSON.stringify({
"type": "session.update",
"session": {
"type": "realtime",
"modelId": "anthropic/claude-sonnet-4-6",
"instructions": "You are a helpful voice agent.",
"output_modalities": ["audio", "text"],
"audio": {
"output": {
"model": "inworld-tts-2",
"voice": "Sarah"
}
}
}
}));Every configuration, one session
Pick any LLM for the conversation engine. Swap providers without changing your integration.
Sub-second response time
Optimized data flow delivers end-to-end speech-to-speech latency under one second. Voice agents respond with human-level cadence.
- Optimized STT, LLM, and TTS pipeline for the best latency and quality.
- Full-duplex audio streaming over WebSocket or WebRTC
Sub-second response time
Optimized data flow delivers end-to-end speech-to-speech latency under one second. Voice agents respond with human-level cadence.
- Optimized STT, LLM, and TTS pipeline for the best latency and quality.
- Full-duplex audio streaming over WebSocket or WebRTC
Intelligent turn taking
Context-aware semantic VAD with adjustable eagerness. The agent knows when to listen, when to speak, and when a user is interrupting.
- Semantic VAD detects intent boundaries, not just silence
- Adjustable eagerness from cautious to aggressive
- Graceful barge-in handling — no awkward overlaps or cut-offs
Intelligent turn taking
Context-aware semantic VAD with adjustable eagerness. The agent knows when to listen, when to speak, and when a user is interrupting.
- Semantic VAD detects intent boundaries, not just silence
- Adjustable eagerness from cautious to aggressive
- Graceful barge-in handling — no awkward overlaps or cut-offs
Conversational intelligence
The inworld/inworld-stt-1 model emits a voice profile — emotion, vocal style, accent, age, pitch — alongside every transcript chunk. Those signals land in the LLM as structured context, the LLM emits Realtime TTS-2 [steering] tags inline, and Realtime TTS-2 renders the response with matching prosody and non-verbal cues.
- inworld/inworld-stt-1 emits 5 paralinguistic signals per audio chunk with confidence scores
- Voice profile flows into LLM context; LLM emits inline [Speak softly] / [sigh] tags
- Realtime TTS-2 consumes the tags and renders expressive audio — no prompt engineering required
Conversational intelligence
The inworld/inworld-stt-1 model emits a voice profile — emotion, vocal style, accent, age, pitch — alongside every transcript chunk. Those signals land in the LLM as structured context, the LLM emits Realtime TTS-2 [steering] tags inline, and Realtime TTS-2 renders the response with matching prosody and non-verbal cues.
- inworld/inworld-stt-1 emits 5 paralinguistic signals per audio chunk with confidence scores
- Voice profile flows into LLM context; LLM emits inline [Speak softly] / [sigh] tags
- Realtime TTS-2 consumes the tags and renders expressive audio — no prompt engineering required
Voice profile steers Realtime TTS-2 in Realtime
The whole pipeline lives in one WebSocket: STT emits a voice profile per chunk, the LLM emits Realtime TTS-2 [steering] tags inline in its response, Realtime TTS-2 renders the response with matching prosody and non-verbal cues like [sigh] and [laugh]. No prompt engineering, no second model call.
- Realtime TTS-2 [Speak softly] / [whisper] / [excited] tags interpreted natively
- Non-verbal cues — [sigh], [laugh], [hmm] — rendered as real audio events
- End-to-end TTFT stays sub-second; the whole pipeline runs as one round-trip
Voice profile steers Realtime TTS-2 in Realtime
The whole pipeline lives in one WebSocket: STT emits a voice profile per chunk, the LLM emits Realtime TTS-2 [steering] tags inline in its response, Realtime TTS-2 renders the response with matching prosody and non-verbal cues like [sigh] and [laugh]. No prompt engineering, no second model call.
- Realtime TTS-2 [Speak softly] / [whisper] / [excited] tags interpreted natively
- Non-verbal cues — [sigh], [laugh], [hmm] — rendered as real audio events
- End-to-end TTFT stays sub-second; the whole pipeline runs as one round-trip
Provider agnostic, full control
Route to hundreds of LLMs, choose your STT engine, and access custom Inworld voices — all from a single session. Swap any component at any time.
- OpenAI, Anthropic, Google, Groq, Mistral, xAI, and more
- Choose STT provider independently of LLM
- Access to all Inworld built-in voices as well as your cloned and custom voices
{"type": "realtime","modelId": "anthropic/claude-sonnet-4-6","stt": { "model": "inworld/stt-1" },"audio": { "voice": "Sarah" }}
Provider agnostic, full control
Route to hundreds of LLMs, choose your STT engine, and access custom Inworld voices — all from a single session. Swap any component at any time.
- OpenAI, Anthropic, Google, Groq, Mistral, xAI, and more
- Choose STT provider independently of LLM
- Access to all Inworld built-in voices as well as your cloned and custom voices
{"type": "realtime","modelId": "anthropic/claude-sonnet-4-6","stt": { "model": "inworld/stt-1" },"audio": { "voice": "Sarah" }}
Fluent tool calling for agents
Register tools at session start or add them on the fly. The assistant calls your functions mid-conversation without breaking the audio stream.
- Declare tools once — the agent invokes them when needed
- Utilize our built-in web search as well as any custom tool you define.
- Audio stays open while tools execute and results stream back
Fluent tool calling for agents
Register tools at session start or add them on the fly. The assistant calls your functions mid-conversation without breaking the audio stream.
- Declare tools once — the agent invokes them when needed
- Utilize our built-in web search as well as any custom tool you define.
- Audio stays open while tools execute and results stream back
Use cases
Inworld Realtime API vs OpenAI Realtime API
Capability | Realtime API | OpenAI Realtime |
|---|---|---|
OpenAI SDK compatible | ||
Sub-second latency | ||
LLM choice | Hundreds of models | GPT-4o only |
TTS quality | #1 ranked TTS on Artificial Analysis | Built-in only |
Custom voices | Built-in + cloned + custom | 6 preset voices |
Function calling | ||
Semantic turn detection | ||
Conversational intelligence | Emotion, age, accent | |
Transport options | WebSocket, WebRTC | WebSocket, WebRTC |
Pricing (per minute) | From $0.015/min | From $0.06/min |
Provider lock-in | None — swap models anytime | OpenAI only |
FAQ
