Skip to main content

How can I reduce latency below 80ms for my AI voice agent calls?

The 80ms target refers to the telephony layer’s contribution to total conversational latency — the audio round trip from the caller’s microphone to your AI platform’s WebSocket and back to the caller’s speaker. Achieving sub-80ms at the telephony layer requires co-located media infrastructure, the right codec, and a direct SIP path without additional routing hops. Vobiz achieves under 80ms audio RTT for India-to-India calls by default.

Understanding AI Voice Latency: The Three Components

Total AI voice agent conversational latency has three parts:
Telephony RTT (target: <80ms): The time for audio to travel from the caller to your AI platform and back. This is controlled by your SIP trunk provider’s infrastructure location, codec, and routing path. AI processing time (100–300ms): Speech-to-text transcription, LLM inference, and response generation. This is determined by your AI platform (Vapi, Retell AI, etc.) and your LLM provider. TTS synthesis time (50–200ms): Converting the LLM’s text response to audio. Determined by your TTS provider (ElevenLabs, Deepgram, etc.) and whether streaming synthesis is enabled. For natural-feeling conversation, the total must stay under ~1,000ms. Keeping telephony RTT at or below 80ms leaves 700–900ms for AI processing + TTS — achievable with modern streaming LLM and TTS stacks.

How to Reduce Telephony RTT

1. Use a Media PoP in the Same Region as Your AI Infrastructure

The single biggest latency contributor is geographic distance between the SIP trunk’s media server and your AI platform’s WebSocket server. If your AI runs in Mumbai (AWS ap-south-1) and your SIP trunk terminates media in Singapore, every audio packet travels an extra 60–80ms each way. Vobiz media PoPs: Mumbai, Delhi, Bengaluru (India); Singapore, US-East, EU-West. When you create a Vobiz Application, select the media region closest to your AI platform’s server location. For AWS ap-south-1 deployments, use the Mumbai PoP.

2. Use G.711 (PCMU/PCMA), Not G.729

G.729 uses algorithmic compression that adds 15–30ms of encoding/decoding latency per hop. G.711 (μ-law or a-law) is an uncompressed codec with near-zero encoding latency. For AI voice agents, always configure your SIP trunk and AI platform to use G.711 PCMU (μ-law, 8kHz). Despite lower audio quality than wideband codecs, G.711 is universally supported and adds no codec latency. For AI platforms that support wideband, use linear PCM at 16kHz or 24kHz — this is higher quality than G.711 and has zero codec latency since no compression occurs.

3. Configure Direct Media (No RTP Proxy)

Some SIP trunk providers relay RTP audio through a proxy server, adding a hop between the caller and your AI platform. Vobiz supports direct media mode where RTP audio flows directly between the caller’s carrier and your AI platform’s media endpoint, bypassing the Vobiz proxy. Enable direct media in your Vobiz Application settings for the lowest possible RTT.

4. Use Vobiz’s WebSocket Audio Stream for Sub-50ms

For the lowest-latency AI voice setup, use Vobiz’s native WebSocket audio stream rather than SIP. When a call arrives at a Vobiz number, Vobiz can open a bidirectional WebSocket to your AI platform and stream raw audio frames directly — no SIP processing overhead. The WebSocket stream delivers audio with 20ms frame intervals. This is the mode used by Vapi and Retell AI when configured with Vobiz as the telephony provider.

5. Reduce Audio Buffer Size on Your AI Platform

AI platforms typically buffer 100–200ms of audio before sending it to STT for transcription. Reducing the buffer to 60–80ms reduces the time before the first word reaches the LLM — but at the cost of some STT accuracy on slower speech. Most AI platforms expose this as a vad_silence_ms or turn_timeout parameter. Test at 60ms and increase if you see incomplete transcriptions.

Measuring Your Current Latency

Alternatively, Vobiz CDRs include rtp_latency_ms per call. Query recent CDRs via the CDR API and filter for high-latency calls to identify patterns.

Expected Latency by Region

Frequently Asked Questions

For India-to-India calls with the Mumbai media PoP, direct media enabled, and G.711 PCMU codec, Vobiz achieves 30–70ms audio RTT. This is the telephony layer contribution only — AI processing and TTS synthesis add further latency on top.
Yes. Vapi and Retell AI add their own processing latency for STT, LLM inference, and TTS — typically 300–700ms combined with optimised models. Vobiz’s telephony latency is a floor, not the ceiling. The total conversational RTT is Vobiz latency + AI platform latency.
For most deployments, the difference is negligible (5–15ms). Use WebSocket if your AI platform recommends it (Vapi and Retell AI both use WebSocket audio internally). Use SIP if you need compatibility with standard SIP endpoints or if your AI platform prefers SIP transport.
Almost certainly not — Vobiz’s telephony RTT is under 100ms by default. A 2-second total response time indicates AI processing is the bottleneck: STT transcription, LLM inference, or TTS synthesis. Profile each stage separately using your AI platform’s latency metrics before tuning the telephony layer.