Voice streaming over WebSockets

Vobiz can fork live call audio to your WebSocket application. Your application can transcribe the caller, run agent logic, generate speech, and send audio back to the caller on the same connection. Use WebSocket streaming when you want to own the STT, agent, TTS, voice-activity detection, barge-in, and conversation state.

How the connection starts

You can create a stream in two ways:

Return a <Stream> XML element from the call’s Answer URL.
Start an Audio Stream through REST on an active call.

For a bidirectional XML stream:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Stream bidirectional="true" keepCallAlive="true">
    wss://your-domain.com/ws
  </Stream>
</Response>

Vobiz connects to your WebSocket, sends a start event, and then sends inbound media events while the call is active.

Treat the audio directions separately

Inbound stream audio
Caller -> Vobiz -> your WebSocket application
Configured by <Stream contentType> or REST content_type
Reported by start.mediaFormat

Outbound playback audio
Your WebSocket application -> Vobiz -> caller
Configured by playAudio.media.contentType and playAudio.media.sampleRate

The two directions can use different supported formats. Always identify a format by its direction instead of using the ambiguous phrase “stream sample rate.”

To play 24 kHz agent audio, send genuine L16/24 kHz data in playAudio with sampleRate: 24000. Do not configure inbound <Stream contentType> or REST content_type as L16/24 kHz for this purpose.

Inbound stream formats

Vobiz reports the selected inbound format in start.mediaFormat.

Encoding	Inbound sample rates
`audio/x-l16`	8000 or 16000 Hz
`audio/x-mulaw`	8000 Hz

Use start.mediaFormat to configure your Base64 decoder, audio processor, or STT service for every connection.

{
  "event": "start",
  "start": {
    "callId": "CALL_ID",
    "streamId": "STREAM_ID",
    "tracks": ["inbound"],
    "mediaFormat": {
      "encoding": "audio/x-l16",
      "sampleRate": 16000
    }
  }
}

Outbound playback formats

Your application sends agent or TTS audio with playAudio.

Encoding	Outbound sample rates
`audio/x-l16`	8000, 16000, or 24000 Hz
`audio/x-mulaw`	8000 Hz

{
  "event": "playAudio",
  "streamId": "STREAM_ID",
  "media": {
    "contentType": "audio/x-l16",
    "sampleRate": 24000,
    "payload": "BASE64_ENCODED_RAW_L16_AUDIO"
  }
}

The payload must be raw mono audio without a WAV, MP3, or other container header. Its real encoding and sample rate must match the metadata in the event.

A 24 kHz outbound payload does not guarantee 24 kHz audio at the handset. The phone-facing carrier, SIP, or PSTN leg may use a lower-rate codec.

WebSocket protocol

Vobiz sends:

Event	Purpose
`start`	Identifies the call and stream and reports the inbound media format.
`media`	Delivers Base64-encoded inbound audio.
`playedStream`	Confirms playback reached a named checkpoint.
`clearedAudio`	Confirms queued playback was cleared.

Your application sends:

Command	Purpose
`playAudio`	Queues outbound playback audio.
`checkpoint`	Marks the end of an utterance for completion tracking.
`clearAudio`	Drops queued playback during barge-in.
`stop`	Ends the stream.

When the call ends, Vobiz closes the WebSocket. Treat the close event as the in-band end-of-stream signal instead of waiting for an inbound stop JSON event.

Voice-agent flow

Vobiz sends start with the inbound media format.
Vobiz sends caller audio in media events.
Your application decodes the payload using start.mediaFormat.
Your STT service transcribes the caller.
Your agent generates a response.
Your TTS service generates audio at a supported outbound rate.
Your application sends raw audio in playAudio events.
Your application sends a checkpoint after the utterance.
Vobiz sends playedStream when playback reaches the checkpoint.

Chunking and barge-in

Send outbound playback in approximately 20–60 ms chunks. This range is recommended for responsive barge-in; it is not a protocol requirement. For 20 ms of mono audio:

Format	Raw payload size
μ-law/8000	160 bytes
L16/8000	320 bytes
L16/16000	640 bytes
L16/24000	960 bytes

When your voice-activity detector confirms that the caller is interrupting, send clearAudio with the active streamId. Audio removed before its checkpoint may not produce a playedStream event, so add a timeout to checkpoint-dependent application logic.

Format conversion

Convert audio only when the next system requires another format:

Inbound
media.payload -> Base64 decode -> decode using start.mediaFormat -> STT format

Outbound
TTS output -> convert to supported raw mono format -> Base64 encode -> playAudio

Do not automatically downsample every TTS response to μ-law/8 kHz. If your application sends supported L16 playback, declare the actual L16 rate in playAudio. If your integration is configured for μ-law/8 kHz, resample and encode the payload before sending it.

Implementation responsibilities

Your application must:

Keep per-call state for each WebSocket connection.
Save callId, streamId, and start.mediaFormat from the start event.
Validate outbound audio before sending it.
Serialize concurrent WebSocket writes.
Clear queued playback when the caller interrupts.
Handle conditional checkpoint acknowledgments and timeouts.
Clean up STT, TTS, buffers, and conversation state on WebSocket close.

Getting Started

Using Vobiz Platform

Compliance (India)

Tools

Voice streaming over WebSockets

How the connection starts

Treat the audio directions separately

Inbound stream formats

Outbound playback formats

WebSocket protocol

Voice-agent flow

Chunking and barge-in

Format conversion

Implementation responsibilities

Next steps

​How the connection starts

​Treat the audio directions separately

​Inbound stream formats

​Outbound playback formats

​WebSocket protocol

​Voice-agent flow

​Chunking and barge-in

​Format conversion

​Implementation responsibilities

​Next steps

How the connection starts

Treat the audio directions separately

Inbound stream formats

Outbound playback formats

WebSocket protocol

Voice-agent flow

Chunking and barge-in

Format conversion

Implementation responsibilities

Next steps