> ## Documentation Index > Fetch the complete documentation index at: https://www.vobiz.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # What Is Voice AI? How It Works, Platforms & Use Cases > Voice AI is software that understands and generates natural human speech in real time — powering AI agents, voice assistants, IVR systems, and call bots. How ASR, LLM, and TTS combine, which platforms lead, and how to build a voice AI system on Vobiz. *November 2026 · Vobiz Team* **Voice AI is software that can hear, understand, and speak — enabling machines to hold real conversations over the phone or through any audio interface.** It powers the AI agents that answer customer calls, the voice assistants that book appointments, and the call bots that run outbound campaigns at scale. Voice AI is not just speech recognition — it's the full pipeline from hearing a caller's words to generating a contextually intelligent spoken response in under a second. > [!NOTE] > > **Key takeaways** > > * Voice AI = ASR (speech recognition) + LLM (reasoning) + TTS (speech synthesis), all running in real time > * End-to-end latency should be under 1,000ms for natural-sounding conversations > * Voice AI platforms (Retell AI, Vapi, ElevenLabs, LiveKit) handle the AI stack; Vobiz handles the telephony layer > * Indian language support (Hindi, Tamil, Telugu, Marathi) is available via Sarvam AI and regional ASR models > * Voice AI replaces rigid IVR menus with open-ended conversational routing ## The Three Layers of Voice AI Every voice AI system — however simple or complex — runs on three layers working in sequence: ``` Caller speaks ↓ 1. ASR (Automatic Speech Recognition) Deepgram / Whisper / Google STT Converts audio → text in real time ↓ 2. LLM (Large Language Model) GPT-4o / Gemini / Claude Understands intent → generates response text ↓ 3. TTS (Text-to-Speech) ElevenLabs / OpenAI TTS / Sarvam Converts response text → natural spoken audio ↓ Caller hears the response (~700ms later) ``` The speed of each layer determines how natural the conversation feels. A human conversational gap is 200–300ms. Voice AI targeting under 1,000ms total latency feels responsive; above 1,500ms starts to feel robotic. ## ASR: Speech Recognition for Voice AI ASR is the entry point — it has to transcribe exactly what the caller said, in real time, in whatever language and accent they're using. Quality varies significantly by provider: | ASR Provider | Latency | Indian Languages | Accuracy (English) | | - | - | - | - | | **Deepgram Nova-2** | \~200ms | Hindi, Tamil | High | | **OpenAI Whisper** | \~300ms | Good multilingual | Very High | | **Google STT** | \~250ms | 10+ Indian languages | High | | **Sarvam AI** | \~300ms | 10 Indian languages | Very High (India) | | **AssemblyAI** | \~250ms | English-focused | High | For India-deployed voice AI handling regional language calls, Sarvam AI or Google STT with Indian language models gives the best accuracy. ## LLM: The Brain of Voice AI The LLM reads the transcribed text, maintains conversation context, decides what action to take, and generates a response. For voice AI specifically, the LLM needs to: * Generate **short, conversational responses** — not paragraphs, but 1–3 sentences that sound natural spoken aloud * **Maintain context** across multiple turns of conversation * **Trigger actions** — look up order status, book appointments, transfer calls * **Handle interruptions** — when a caller talks over the response, the system should stop and listen **Model selection for voice AI:** | Model | Latency | Cost | Best for | | - | - | - | - | | GPT-4o-mini | \~200ms | Low | High-volume, cost-sensitive | | GPT-4o | \~400ms | High | Complex reasoning, enterprise | | Gemini 1.5 Flash | \~200ms | Low | Speed-critical deployments | | Claude Haiku | \~200ms | Low | Structured conversations | | Llama 3 (hosted) | \~300ms | Very low | Self-hosted, data-sensitive | ## TTS: Giving Voice AI Its Voice TTS converts the LLM's text response to speech. Voice quality here directly determines how human the interaction feels: | TTS Provider | Voice Quality | Latency | Indian Languages | | - | - | - | - | | **ElevenLabs** | Ultra-realistic | \~100ms (streaming) | Limited | | **OpenAI TTS** | Natural | \~100ms | Moderate | | **Sarvam AI** | Natural (Indian accent) | \~150ms | Hindi, Tamil, Telugu etc | | **Google TTS** | Good | \~80ms | 10+ Indian languages | | **Azure TTS** | Good | \~90ms | Multiple Indian | For Indian-market voice AI, Sarvam AI TTS produces the most natural-sounding output for Hindi and regional languages — an Indian-accented English TTS sounds more trustworthy to Indian callers than an American-accented one. ## Voice AI vs Traditional IVR | Factor | Traditional IVR | Voice AI | | - | - | - | | **Caller interaction** | "Press 1 for sales" | "How can I help you today?" | | **Input type** | DTMF keypad | Natural speech | | **Menu depth** | Fixed (usually 2–3 levels) | Unlimited | | **Handling unexpected input** | "I didn't understand that" | Context-aware clarification | | **Personalisation** | None | CRM-driven, caller-specific | | **Language support** | Pre-recorded audio | 20+ languages in real time | | **Setup** | Pre-record all audio | Write a system prompt | | **Updates** | Re-record audio | Edit the prompt | Voice AI doesn't eliminate IVR — for simple, high-volume interactions (OTP delivery, appointment reminders), a DTMF IVR is cheaper and faster. Voice AI is the right choice when the conversation needs to be flexible, personalised, or multi-step. ## Voice AI Use Cases **AI receptionist**: Answers all inbound calls, greets callers, routes based on spoken intent, books appointments, handles FAQs. Runs 24/7 with no queue wait. See [What is an AI Receptionist](https://www.vobiz.ai/docs/blogs/what-is-an-ai-receptionist.md). **Outbound sales calls**: AI agent dials prospect lists, delivers a natural-sounding pitch, handles objections using LLM reasoning, qualifies leads, books demos, logs outcomes to CRM. See [AI Cold Calling](https://www.vobiz.ai/docs/blogs/what-is-ai-cold-calling.md). **Customer support**: Answers common questions (order status, return policy, account balance) without connecting to a human agent. Escalates complex issues with full context summary. **Post-call surveys**: Calls customers after a service interaction, asks 2–3 satisfaction questions in natural speech, transcribes responses, and feeds NPS/CSAT data to analytics dashboards. **Appointment reminders**: Calls patients or customers before their appointment, confirms attendance, reschedules if needed — all without human involvement. **Collections and payment reminders**: Calls on overdue accounts, reads the outstanding balance, offers payment options, connects to payment support if requested. ## Voice AI Platforms Voice AI platforms provide the managed ASR + LLM + TTS pipeline with pre-built developer tools. The telephony layer (connecting to real phone numbers) is a separate component, typically provided by Vobiz via SIP trunking. | Platform | Best for | Connection to Vobiz | | - | - | - | | **Retell AI** | No-code agent builder | SIP trunk | | **Vapi** | Developer-first, function calling | SIP trunk | | **ElevenLabs** | Ultra-realistic voice quality | SIP trunk | | **LiveKit** | Real-time media, custom pipelines | WebSocket / SIP | | **Deepgram** | ASR-layer, custom LLM | WebSocket | | **Ultravox** | Low-latency, open-source | WebSocket | | **Pipecat** | Python framework, custom stack | WebSocket | | **Sarvam AI** | Indian languages | WebSocket | All of these platforms connect to Vobiz to access real phone numbers and PSTN routing. The AI platform handles the conversation; Vobiz handles the call. ## Building a Voice AI System on Vobiz **Architecture:** ``` Phone number (Vobiz) ↓ Inbound call arrives SIP trunk (Vobiz) ↓ Routes to AI platform Voice AI Platform (Retell / Vapi / Custom) ↓ ASR → LLM → TTS Real-time conversation ↓ Call ends Webhook → CRM / Analytics ``` **Step 1**: Get a phone number on Vobiz — Indian 10-digit, 140-series, or international DID. **Step 2**: Choose a voice AI platform and create your agent (system prompt, voice, language). **Step 3**: Create a Vobiz SIP trunk and connect it to the platform. For inbound calls, create an inbound trunk pointing to the platform's SIP URI. For outbound, create an outbound trunk with digest auth credentials. **Step 4**: Test by calling your Vobiz number. The AI agent should answer, understand your speech, and respond naturally. Detailed setup guides: * [Retell AI + Vobiz](https://www.vobiz.ai/docs/integrations/retellai-dashboard.md) * [Vapi + Vobiz](https://www.vobiz.ai/docs/integrations/vapi-dashboard.md) * [ElevenLabs + Vobiz](https://www.vobiz.ai/docs/integrations/elevenlabs-dashboard.md) * [LiveKit + Vobiz](https://www.vobiz.ai/docs/integrations/livekit.md) * [Sarvam AI + Vobiz](https://www.vobiz.ai/docs/integrations/sarvam.md) ## Latency Optimisation for Voice AI The goal is under 1,000ms end-to-end. Each layer contributes: | Layer | Baseline | Optimised | | - | - | - | | ASR | 300ms | 150ms (streaming ASR) | | LLM inference | 500ms | 200ms (smaller/faster model) | | TTS | 200ms | 80ms (streaming TTS) | | Network | 150ms | 50ms (PoP in India) | | **Total** | **1,150ms** | **480ms** | The key optimisation is **streaming**: start TTS as soon as the LLM generates the first few words, rather than waiting for the full response. Vobiz's Mumbai infrastructure reduces network latency for India-originated calls. ## India Considerations for Voice AI **Language**: Hindi and 10 Indian regional languages are supported across different ASR/TTS combinations. See the [Sarvam AI integration](https://www.vobiz.ai/docs/integrations/sarvam.md) for the most comprehensive Indian language coverage. **TRAI compliance**: Outbound voice AI calls to Indian consumers must use 140-series numbers, be DLT-registered, and honour NDNC/DND preferences. See [India calling compliance](https://www.vobiz.ai/docs/compliance/india/calling-regulations.md). **Latency**: Vobiz's infrastructure is based in Mumbai, keeping round-trip latency for India calls under 100ms on the network layer — critical for sub-second total response time. **Caller trust**: Indian callers often disconnect if an AI agent sounds foreign. Using an Indian-accented voice (Sarvam TTS, Google TTS with Hindi voices) significantly improves engagement rates. ## Next Steps * [Explore AI integrations](https://www.vobiz.ai/docs/integrations.md) — Retell, Vapi, ElevenLabs, LiveKit and more * [Build an AI voice agent](https://www.vobiz.ai/docs/solutions/ai-voice-agent.md) — full architecture guide * [Read about AI receptionist setup](https://www.vobiz.ai/docs/blogs/what-is-an-ai-receptionist.md) * [Get a Vobiz phone number](https://console.vobiz.ai) to put your voice AI on a real line This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.