Skip to main content
November 9, 2026 · By Piyush Sahoo A voice agent is an AI system that makes and receives phone calls, understands what the other person says, and responds in natural speech — without a human on the agent side. It is the AI equivalent of a call center agent, but it operates at any scale, never sleeps, and costs a fraction of a human per interaction. Voice agents are distinct from chatbots (which communicate by text) and from traditional IVRs (which follow fixed menus). They conduct natural, open-ended conversations — handling follow-up questions, context shifts, and unexpected input that neither a chatbot nor an IVR can manage.
Key takeaways
  • A voice agent = STT (listen) + LLM (understand and decide) + TTS (speak), connected to telephony (phone network).
  • Inbound voice agents answer calls; outbound voice agents make calls.
  • The differentiator from IVR is natural language understanding — no “press 1 for sales”.
  • Latency under 1,500 ms is required for a conversation that feels natural.
  • For India: inbound voice agents work on any DID; outbound agents need 140-series numbers.

​
What is a voice agent?

A voice agent is a software system that participates in a phone call as an active conversational party. It listens to audio from the other end of the call, converts that audio to text using speech recognition, generates a response using a language model, converts that response back to audio using text-to-speech, and plays it into the call — all in real time, in under 1–2 seconds per turn. The result is a system the caller experiences as talking to a person — it responds to what they say, remembers what was said earlier in the call, asks follow-up questions, and handles unexpected phrasing.

​
The four components of a voice agent

1. Telephony (the phone line) The voice agent needs a way to make and receive actual phone calls. This requires:
  • A phone number (DID for inbound; 140-series for outbound A2P in India)
  • A SIP trunk that routes calls from the phone network to your AI pipeline
  • A bidirectional audio stream so your pipeline can send synthesized speech back into the call
Vobiz provides all three. When a call connects, Vobiz streams audio to your AI pipeline via WebSocket and plays back whatever audio your pipeline returns. 2. Speech-to-text (the ears) STT converts the caller’s audio to text. Modern streaming STT sends partial transcripts while the caller is still speaking — this is essential for low-latency responses. The STT needs to handle:
  • Background noise from phone calls (8 kHz narrowband audio on PSTN)
  • The caller’s accent and speaking style
  • Domain-specific vocabulary (medical terms, product names, etc.)
3. Large language model (the brain) The LLM processes the caller’s transcript, the conversation history, and your system prompt — which defines the agent’s role, knowledge, tone, and decision rules — and generates the agent’s next response. The system prompt is where the voice agent’s “personality” lives. It tells the LLM what the agent is called, what it knows, what it can and cannot do, and how to handle edge cases. 4. Text-to-speech (the voice) TTS converts the LLM’s text response to audio. Streaming TTS starts playing audio before the full sentence is synthesized, cutting perceived latency. Voice selection (tone, accent, gender) affects how the agent is received — most TTS providers offer dozens of voice options.

​
Inbound vs. outbound voice agents

Inbound voice agents answer calls that come to your number. Use cases: customer service, appointment booking, FAQ handling, after-hours coverage, lead capture. Outbound voice agents place calls to a contact list. Use cases: appointment reminders, payment follow-ups, surveys, delivery confirmations, sales outreach. Outbound agents require answering machine detection (AMD) to confirm a human answered before starting the conversation.

​
How voice agents handle a conversation

A typical inbound conversation with a voice agent:
Each turn takes 500–1,500 ms. The caller experiences it as a natural conversation.

​
Handling interruptions and edge cases

Barge-in. If the caller starts speaking while the agent is talking, the agent should detect this and stop speaking — responding to what the caller said rather than finishing its previous sentence. This requires the orchestration layer to monitor inbound audio while TTS is playing and interrupt the TTS stream when the caller speaks. Long silence. If the caller doesn’t speak for a few seconds, the agent should prompt: “Are you still there?” or assume the call ended and hang up gracefully. Out-of-scope requests. If the caller asks for something outside the agent’s defined role, the system prompt should instruct the LLM to acknowledge the limit and either escalate to a human or explain what it can help with. Failed understanding. If the STT returns a low-confidence transcript, the agent should ask the caller to repeat — not guess and proceed incorrectly.

​
Building a voice agent with Vobiz

No-code path (Vapi or Retell AI):
  1. Create a Vobiz account and get a phone number
  2. Set up your AI agent in Vapi or Retell AI (system prompt, voice, knowledge)
  3. Connect Vobiz as the telephony provider via the dashboard
  4. Your agent is ready to make and receive calls
Time to first call: 15–30 minutes. See Vapi dashboard guide or Retell AI dashboard guide. Custom build (Vobiz WebSocket stream): Configure your Vobiz number’s webhook to return a <Stream> element:
Your WebSocket server receives 20 ms audio frames (event: media), processes them through your STT → LLM → TTS pipeline, and sends audio back via playAudio events. For a full example, see the LiveKit inbound agent template or the Pipecat integration guide.

​
Voice agents for India

India deployments have specific requirements: Language. India has diverse accents in English and multiple regional languages. Test your STT on actual India caller audio, not just US-accented test sets. For Hindi and other Indian languages, Sarvam AI provides India-optimized STT and TTS that integrates natively with Vobiz. Outbound compliance. Outbound voice agents calling Indian numbers must use the 140-series number format and comply with TRAI’s automated calling rules. Vobiz provisions 140-series numbers with the right sub-series for your use case (service, promotional, survey). Latency. India’s carrier network can introduce additional jitter on some routes. Use a SIP trunk provider with direct India carrier relationships — not one routing through Singapore. Vobiz has direct interconnects with Jio, Airtel, and BSNL.

​
Voice agent platforms and tools

For a step-by-step guide to building your first voice agent on Vobiz, see How to Build an Inbound AI Voice Agent or How to Build an Outbound AI Voice Agent.