Skip to main content
Voice AI agents are only as good as the audio they receive. When background noise, packet loss, or narrowband codecs degrade the audio stream, ASR word error rates rise, the LLM misunderstands intent, and the conversation breaks down — before the AI ever gets a chance to respond. Vobiz’s media layer is built to prevent this. Every call passes through a stack of audio processing capabilities designed specifically for voice AI pipelines.

​
HD Voice (Wideband Audio)

Standard telephone networks use narrowband codecs (G.711 at 8 kHz), which reproduce human speech in the 300 Hz–3.4 kHz range. This was designed for voice intelligibility on PSTN copper lines — not for ASR accuracy. Vobiz supports wideband and ultra-wideband audio through G.722 (50 Hz–7 kHz) and Opus (up to 20 kHz, variable bitrate). For AI voice agent deployments: Why it matters for AI agents: ASR models are trained on wideband audio. Feeding them narrowband G.711 audio forces them to work with half the frequency information — WER increases measurably, especially for consonants (s, f, th) that live in the 3–8 kHz range. When streaming audio to Deepgram, Whisper, or Sarvam AI via WebSocket, Vobiz streams Opus at 16 kHz+ for maximum transcription accuracy.

​
Voice Activity Detection (VAD)

Voice Activity Detection distinguishes active speech from silence and background noise at the packet level. Vobiz applies VAD continuously on every media stream. What VAD enables: Efficient streaming: Silent packets are suppressed (Comfort Noise Generation replaces them) — reducing bandwidth and processing load on the AI agent’s WebSocket server by up to 50% on typical calls where callers pause frequently. Turn-taking signals: When a caller stops speaking, VAD fires an end-of-utterance signal to the AI agent. This is the trigger for the ASR to finalise the transcript and pass it to the LLM. Faster, accurate VAD = lower perceived latency between caller and AI response. Interruption detection: When the caller speaks while the AI is responding, VAD detects the interruption and signals the agent platform to stop TTS playback and start listening. This is what makes AI conversations feel interruptible rather than robotic. AMD integration: Vobiz’s Answering Machine Detection engine uses VAD as its primary signal — machine greetings and human responses have distinct VAD patterns that drive the human/machine classification.

​
Noise Cancellation

Vobiz applies media-layer noise suppression on inbound audio before it reaches the AI agent’s ASR. This is distinct from client-side noise cancellation (which happens in the caller’s microphone or device) — it operates on the received audio stream at the Vobiz media server. What it handles:
  • Stationary noise: HVAC hum, fan noise, office background noise — continuous spectral patterns are suppressed using adaptive filtering
  • Non-stationary noise: Traffic, construction, busy environments — handled via spectral subtraction on the media stream
  • Echo: Acoustic echo from loudspeakers on the caller’s device is cancelled before the signal enters the ASR pipeline
Why this matters specifically for AI agents: Most ASR models handle clean speech well. Their performance degrades significantly on noisy audio. By the time the signal reaches Deepgram or Whisper, it should be as clean as possible. Noise cancellation at the media layer does this universally — regardless of which ASR the agent uses — without requiring every AI platform integration to implement its own denoising. A caller on a busy street or in a noisy call centre can produce audio where up to 40% of the spectral energy is noise. Media-layer noise cancellation recovers intelligible speech from these environments.

​
Voice Isolation

Voice isolation goes a step further than noise cancellation. Where noise cancellation suppresses background spectral energy, voice isolation identifies and extracts only the primary speaker’s voice from the audio stream — separating it from secondary speakers, crosstalk, and overlapping audio sources. This is applied at the Vobiz media layer using a neural separation model that runs in real time on the inbound audio stream. What voice isolation handles:
  • Multi-speaker environments: Contact centre floors where other agents are audible in the background — voice isolation locks onto the primary caller’s voice and strips everything else
  • Crosstalk: Family members, colleagues, or bystanders speaking near the caller — their voices are separated and removed before the stream reaches ASR
  • TV / radio in background: Broadcast speech that narrowband noise suppression struggles to distinguish from human voice — voice isolation identifies it as a non-primary source and suppresses it
  • Call-centre agent background noise: For outbound agents calling from a noisy contact centre environment, voice isolation ensures only the target contact’s voice reaches the AI pipeline
How it differs from noise cancellation: The two work in sequence: noise cancellation removes the stationary and non-stationary noise floor first; voice isolation then separates the primary voice from any remaining speech signals. Why this matters for AI agents in real-world deployments: Many enterprise voice AI use cases — patient intake in a clinic waiting area, customer support from a shared workspace, field sales calls in an open office — involve calls where background speech is present. Without voice isolation, the ASR may transcribe bystander speech or background conversation fragments alongside the caller’s words, introducing hallucinated text into the transcript and confusing the LLM. Voice isolation ensures the transcript represents only what the caller said.

​
Packet Loss Concealment (PLC)

Voice calls travel over UDP — which is connectionless and has no retransmission mechanism. On any real-world network path, some RTP audio packets will be lost: mobile handoffs, congested ISP links, WiFi interference. Without PLC, lost packets produce brief but audible dropouts — clipping and gaps in the audio stream. For human callers, a 1–2% packet loss rate is barely noticeable. For ASR models, the same dropout rate can cause word-level transcription errors because the model receives a discontinuous input it wasn’t trained on. Vobiz implements PLC on the media server using predictive synthesis:
  1. When a packet loss is detected (gap in RTP sequence numbers), PLC generates a synthetic replacement packet
  2. The replacement is derived from the recent audio waveform — a short extrapolation of the pitch and formant structure from the preceding frames
  3. The synthetic packet is seamlessly inserted into the stream before it reaches the AI agent
Effect on voice AI: On a call with 2% packet loss, PLC typically reduces ASR errors attributable to dropouts by 70–80%. The ASR receives a continuous, smooth audio stream even when the network path is imperfect.

​
Forward Error Correction (FEC)

FEC is a proactive mechanism — rather than recovering lost packets after the fact (like PLC), it embeds redundant audio data in future packets so that lost packets can be reconstructed without synthesising a replacement. Vobiz uses Opus FEC on WebSocket audio streams to AI agents:
  • The encoder embeds a lower-bitrate copy of the current audio frame into the next RTP packet
  • If the current packet is lost, the redundant data in the next arriving packet reconstructs it exactly
  • If the current packet arrives, the redundant data is discarded — zero overhead on clean networks
FEC vs PLC — complementary, not competing: FEC activates first. PLC handles what FEC cannot recover — burst losses of 2+ consecutive packets, or loss of the packet carrying the FEC data itself. Together, the two mechanisms protect intelligibility across the range of real-world network conditions — from isolated single-packet drops (FEC) to multi-packet burst loss on degraded mobile paths (PLC).

​
Combined Impact on Voice AI

For AI voice agent deployments, these six layers work together:
The practical outcome: an AI voice agent receiving audio processed through this stack achieves measurably lower WER on the same underlying network conditions compared to raw RTP delivery.

​
Codec Configuration for AI Agents

SIP trunk connections (Retell AI, Vapi, ElevenLabs): Vobiz negotiates the highest quality codec supported by both endpoints during the SIP INVITE/200 OK handshake. For AI platforms that support G.722 or Opus, those are preferred over G.711. WebSocket streaming connections (Pipecat, LiveKit, custom pipelines): When using the <Stream> XML verb, the audio format is configured in the stream parameters:
For AI pipelines that accept raw PCM or L16, specify accordingly:
16 kHz L16 gives ASR models the best input for high-accuracy transcription. The noise cancellation and PLC stack processes the audio before it reaches your WebSocket endpoint regardless of the codec selected.

​
Why This Architecture Was Built

Most CPaaS providers route media through general-purpose infrastructure — the same media servers handle calls for IVR systems, conference bridges, SMS, and voice calls alike. Audio quality features are bolt-ons. Vobiz’s media layer was designed specifically for AI voice agent workloads, where the audio that reaches the ASR is a first-order concern. The noise cancellation, VAD, PLC, FEC, and HD codec support are not optional modules — they run on every call in the Vobiz infrastructure. For builders deploying AI agents in real-world environments — customer support lines that receive calls from noisy factories and moving vehicles, outbound campaigns dialling to mobile numbers on variable-quality networks — this infrastructure layer is the difference between a voice AI that works reliably and one that works only in controlled conditions.