The Telephony Layer for AI Voice Agents: Connecting Vapi, Retell, ElevenLabs, LiveKit, Pipecat & More to Phone Numbers

A voice agent that reasons instantly and sounds genuinely human is still just an idea sitting inside an API response until something wires it into a phone number a customer can actually dial. That wiring is the telephony layer: the carrier infrastructure and signaling stack that sits between a platform like Vapi, Retell AI, or ElevenLabs and the traditional phone network, handling call routing, number provisioning, and the audio transport that makes a phone call sound like a phone call.

This guide walks through connecting a number to Vapi, Retell AI, ElevenLabs, LiveKit, and Pipecat, and rounds up the dozen-plus other platforms and connection paths Vobiz supports.

What Is the Telephony Layer for AI Voice Agents?

An AI voice agent platform handles the intelligence: transcription, reasoning, and speech generation. It does not, on its own, own a phone number or a connection to the public phone network. That's the job of the telephony layer, sometimes called the carrier layer, and it's made up of a few concrete pieces, such as: a DID (direct inward dial number, the actual 10-digit or toll-free number a caller dials), a SIP trunk (the virtual line that carries the call as data instead of a circuit-switched signal), and the signaling and media servers that route audio between the public switched telephone network (PSTN) and whatever platform is running the AI agent.

Vapi, Retell AI, and ElevenLabs all ship with their own native numbers you can buy directly from them. The telephony layer becomes a separate decision the moment a business wants a number from a specific country, wants to keep an existing number, needs volume pricing a platform's native numbers don't offer, or wants one carrier serving multiple AI platforms and internal tools at once.

How AI Voice Agents Connect to Phone Numbers

The path a phone call takes from a ringing number to an AI-generated response follows the same handful of steps regardless of which platform is listening on the other end.

1. A caller dials a DID. The number lives with a telephony carrier, not with the AI platform itself.
2. The carrier's SIP trunk answers the call. It converts the incoming PSTN circuit into a SIP session, the protocol that carries call setup, teardown, and routing instructions over the internet.
3. The SIP session hands off to the AI platform. For Vapi, Retell AI, ElevenLabs, and LiveKit, this happens directly over a SIP trunk: the carrier sends the call to a SIP address the platform exposes, and RTP (real-time transport protocol) packets carry the actual audio both ways. Pipecat, an open-source framework rather than a hosted platform, typically takes this a step further: the carrier requests instructions from a server the developer controls, and that server opens a WebSocket where raw audio streams bidirectionally to a bot process running locally or in the cloud.
4. The platform's pipeline processes the audio. Speech-to-text (engines like Deepgram or OpenAI's Whisper) turns the caller's voice into text, a language model (GPT-4o, Claude, or a fine-tuned equivalent) generates a response, and a text-to-speech engine (ElevenLabs, Cartesia, or similar) turns that response back into audio.
5. The generated audio streams back over the same SIP/RTP or WebSocket path the call came in on, and the caller hears a reply.
6. Call metadata gets logged on the telephony side: recordings, transcripts, and quality metrics like jitter, packet loss, and mean opinion score, which matter more for diagnosing a bad call than anything happening inside the AI pipeline.

The telephony hop itself is usually the fastest link in that chain. Vobiz, for example, publishes sub-80ms latency on its voice network; the AI processing steps, speech-to-text, reasoning, and speech generation, typically add more time to a response than the network connection that carries it.

Ready to wire your agent? Connect your Vapi, Retell, or custom voice stack to global PSTN numbers with sub-80ms latency. Get started with Vobiz for free.

Connection Methods Compared

Every AI voice agent platform in this guide supports at least one of three connection methods, and most support two.

Why the Telephony Layer Matters

Gartner predicts conversational AI will cut contact center agent labor costs by $80 billion in 2026, and the firm has been tracking the same trend since 2022, which suggests it wasn't a one-year spike. Separately, Market.us projects the voice AI agents market will grow from $2.4 billion in 2024 to $47.5 billion by 2034. Neither number means much without a phone number to answer, though; a voice agent that only works in a browser demo doesn't reduce labor costs or generate revenue.

The telephony layer is what turns a voice agent from a demo into a channel a business actually runs on:

• Outbound sales and collections dialing, where the same agent needs to call out from a local number in whatever country or region it's reaching.
• Inbound support and appointment lines, where a business keeps an existing published number and routes it to an AI agent instead of a human queue.
• IVR replacement, swapping a menu tree for a conversational agent on a number customers already know.
• Multi-platform routing, where a marketing team runs campaigns from one number, support from another, and outbound sales from a third, all provisioned from a single carrier account.

Connecting Vapi to a Phone Number

Vapi is a developer-focused platform for building production voice agents, with a dashboard workflow that covers most setup without code. Connecting a telephony provider's SIP trunk is how a Vapi agent gets a number outside Vapi's own inventory, whether that's a specific country's local number or an existing business line.

Outbound calling is dashboard-only: add a SIP trunk credential in Vapi using the provider's SIP domain, username, and password, import the number as a bring-your-own-SIP-trunk number, assign it to an assistant, and place the call. No code required at any point.

Inbound calling needs one API call. Vapi requires an inbound SIP trunk configured with a gateway for each of the provider's signaling IPs on UDP port 5060, and the resulting trunk ID isn't shown in the dashboard, it has to be retrieved through Vapi's API. From there, a matching inbound trunk gets created on the carrier's side, the number gets linked to it, and an answering assistant gets assigned to the number.

What this unlocks in practice: a Vapi agent built for a US-based product can answer on a US toll-free number and also dial out on a local Indian mobile number for a regional sales motion, all from one carrier account rather than juggling multiple platform-native number pools. Our own Vapi integration page, for instance, exposes both flows through its console alongside coverage across 130+ countries.

Connecting Retell AI to a Phone Number

Retell AI is built around no-code conversational agents, and its telephony integration follows the same philosophy: SIP trunk setup lives entirely in dashboards on both sides, no webhooks or custom answer URLs required.

The architecture uses two trunks moving in opposite directions from a single phone number. Outbound calls authenticate from Retell into the carrier using SIP credentials entered directly in the Retell dashboard: the carrier's phone number, its SIP domain as the termination URI, and a trunk username and password. Inbound calls flow the other way, with the carrier's console configured to route to Retell's SIP hostname over UDP.

One number can carry different agents in each direction, an outbound sales agent placing calls and a separate inbound support agent answering them, without maintaining an IP allowlist or a webhook endpoint for either side. Retell also merges telemetry from both layers, pairing its own conversation transcripts with the carrier's call-quality metrics for QA that spans the full call rather than just the AI portion of it.

We describe our own Retell AI setup as "five fields and a Save" for the outbound side, which is a fair description of how thin dashboard-based SIP setup has gotten across the category.

Connecting ElevenLabs to a Phone Number

ElevenLabs built its name on realistic text-to-speech, and its Conversational AI agents connect to telephony providers through a dashboard flow with no code required on either the outbound or inbound side.

Outbound: import a phone number from the SIP trunk into ElevenLabs, label it, configure outbound settings with the trunk's transport type (typically TCP) and SIP address, and place a test call.

Inbound: configure the SIP URI in the carrier's trunk settings, import the same number into ElevenLabs, attach it to a conversational agent, and dial the number to verify.

The credential set ElevenLabs asks for is small and consistent across most SIP-based carriers: transport type, SIP trunk address (a domain like <account>.sip.<provider>.ai), and a username and password pulled from the carrier's console. Once connected, ElevenLabs' built-in analytics, call logs, transcripts, and recordings, apply to calls made over the SIP trunk the same way they would to a native ElevenLabs number, so switching to a BYOC number doesn't mean losing visibility into call quality. Check out Vobiz ElevenLabs integration for more information.

Connecting LiveKit to a Phone Number

LiveKit is an open-source, self-hostable real-time infrastructure project, which makes its telephony setup slightly more explicit than the fully-hosted platforms above, though still dashboard-driven on the carrier side.

Setup starts in the carrier's console: create a SIP trunk, name it, and generate authentication credentials. Three values come out of that step: the SIP domain (gateway URL), a username, and a password. On the LiveKit side, those map directly into trunk configuration fields, address, auth_username, and auth_password, alongside LiveKit's own URL, API key, and API secret. Inbound calling additionally requires the number itself to be configured in LiveKit's inbound trunk settings.

Because LiveKit is closer to raw infrastructure than a finished product, it's a common choice for teams building a custom voice pipeline rather than using a hosted assistant builder. Our own documented LiveKit setup runs roughly 15 minutes end to end and is explicitly scoped as beginner-friendly with no code, despite LiveKit's developer-first positioning.

Connecting Pipecat to a Phone Number

Pipecat is an open-source Python framework for building real-time, multimodal voice pipelines, and unlike the platforms above, it has no dashboard at all. Every piece of the telephony connection is configured in code and environment variables.

The flow: an outbound call gets triggered through a REST API call to the carrier, which requests call-handling instructions (typically XML) from a server the developer runs. That server responds with a WebSocket URL, and bidirectional audio streams between the carrier's infrastructure and the Pipecat bot process from that point on, whether it's running locally or in the cloud.

Setup requires a handful of credentials: an API key for whichever model provider is handling language and speech (OpenAI is common, with Deepgram as an alternative for speech-to-text), the carrier's account ID and auth token, and a public URL, typically an ngrok tunnel during development, so the carrier can reach a server running on a developer's machine. A working local setup needs Python 3.10 or newer, and our documented example gets a Pipecat bot answering real calls in about five minutes once ngrok is running. Check out Vobiz Pipecat integration for more information.

More AI Voice Platforms You Can Connect

Vapi, Retell AI, ElevenLabs, LiveKit, and Pipecat cover most of the AI voice agent traffic today, but the same SIP-trunk-or-WebSocket pattern extends to a longer list of platforms Vobiz already supports. The setup mechanics stay consistent with what's above: dashboard-based SIP credentials for hosted platforms, code-based WebSocket or REST connections for open-source and developer-first ones.

Each of these follows the same underlying logic covered above: a carrier hands off audio either as a SIP/RTP session or as a raw WebSocket stream, and the platform's own dashboard or SDK decides which one it expects.

Connecting Existing Phone Systems and Browser-Based Calling

Not every deployment starts from a hosted AI platform. Some businesses need a voice agent to plug into infrastructure they already run, or need a way to test a trunk before wiring it into a live agent.

3CX, a business PBX system, connects to a SIP trunk the same way an AI platform does, which makes it possible to route a subset of extensions on an existing office phone system to an AI agent without replacing the PBX itself. Zoiper, a SIP softphone, serves a narrower but useful purpose: testing trunk credentials and call routing manually before pointing a live number at an AI agent, which catches configuration mistakes before a real caller does.

For products that don't need a traditional phone number at all, WebRTC lets a browser or an app place and receive calls directly, embedding a softphone into a website or product with a phone number attached instantly rather than provisioned through a separate telephony console. WebSockets audio streaming is the same raw, bidirectional audio transport Pipecat uses, available on its own for any custom voice pipeline that doesn't fit a platform's native SIP or WebRTC support.

How to Choose a Telephony Layer for Your AI Voice Agent

Every platform above will accept a SIP trunk or a WebSocket connection from more or less any carrier that follows standard protocols. The differences that actually matter show up in six places.

Latency: The telephony hop should be the least noticeable part of a call. Sub-80ms carrier latency, the figure Vobiz publishes for its own voice network, is small enough that speech-to-text, reasoning, and speech generation will each add more delay to a response than the network connection carrying it.

Geographic coverage and number types: A platform that only offers US numbers natively becomes a real constraint the moment a business needs an Indian mobile number, a UK landline, or a toll-free line in more than one country at once. Coverage across 130+ countries for inbound DIDs and 190+ for outbound dialing is a reasonable bar for a global GTM motion.

BYOC support versus formal number porting: Most AI platforms support bringing a carrier's SIP trunk in as a BYOC number faster than they support a formal port-in of an existing number from another carrier, which can take days regardless of provider. Confirming which one a business needs before committing to a timeline avoids a late surprise.

Connection method flexibility: SIP, WebSocket, and REST API support across the same account means one carrier can serve a no-code platform like Retell AI and a custom Pipecat build without separate contracts.

Security: SRTP for media encryption and TLS for signaling should be table stakes, not an upsell, for any call carrying customer conversations.

Developer tooling: SDKs across the languages a team already uses (Python, Node.js, Go, Java, C#) shorten the distance between "SIP trunk connected" and "call flow actually shipped."

Conclusion

Picking a telephony layer isn't a footnote to picking an AI voice agent platform. It's the decision that determines whether that agent can actually answer a phone in the countries a business operates in, at the call volume it needs, without becoming a second vendor relationship to manage. Vobiz documents the connection paths above for Vapi, Retell AI, ElevenLabs, LiveKit, Pipecat, and the rest of its integration library, along with SDKs, WebRTC, and REST API options. Explore the full Vobiz integration library, or start a free trial to connect a number to an existing agent directly.

FAQs

What is the difference between a SIP trunk and a phone number for an AI voice agent?

A phone number (a DID) is the address a caller dials. A SIP trunk is the connection that carries the call as data between the carrier that owns that number and whatever platform is running the AI agent. A platform needs both: a number to be reachable at, and a trunk to actually receive the audio.

Do I need a SIP trunk to connect Vapi, Retell AI, or ElevenLabs to a phone number?

Only if the number isn't coming from the platform's own native inventory. All three platforms sell numbers directly, but connecting a number from an outside carrier, for country coverage, existing business lines, or volume pricing, requires configuring that carrier's SIP trunk in the platform's dashboard.

What is BYOC and how is it different from number porting?

BYOC (bring your own carrier) means connecting an outside carrier's SIP trunk to an AI platform while the number itself stays with that carrier. Number porting formally transfers the number's ownership from one carrier to another, which typically takes longer and isn't supported by every carrier for every country.

How much latency does the telephony layer add to an AI voice call?

Carrier-grade telephony infrastructure typically adds the least latency of any step in the call, well under the delay introduced by speech-to-text, language model reasoning, and speech generation combined. Vobiz, for example, publishes sub-80ms latency on its voice network.

Can I use the same phone number for both inbound and outbound AI voice calls?

Yes. Retell AI and Vapi both support linking a single number to separate inbound and outbound trunks, which means one number can run a different agent in each direction, an outbound sales agent and an inbound support agent, without needing two numbers.

Which AI voice agent platforms support WebSocket instead of SIP?

Open-source frameworks like Pipecat are built around WebSocket streaming by default, since there's no hosted dashboard managing the SIP layer for you. LiveKit supports both SIP and WebSocket, depending on whether a team wants a managed trunk or a fully custom audio pipeline.

Is SIP trunking secure enough for regulated industries like healthcare or finance?

SIP trunking itself is just a transport protocol; security depends on the carrier's implementation. Look for SRTP encryption on the media stream and TLS on the signaling side as a baseline before treating a SIP trunk as production-ready for regulated call traffic.

Vobiz

Navigating carrier infrastructure can feel daunting when your main focus is building great conversational experiences. Different platforms require different connection paths, from dashboard SIP credentials to code-based WebSocket streams.

We published a clear guide to help simplify this process. It walks through connecting phone numbers to major voice platforms, comparing setup steps, latency considerations, and connection methods side by side.

If you are currently wiring up telephony for your voice agent, we hope this guide makes the setup straightforward and hassle-free.

Comment

Read the guide here: {link}

Meta title (≤60 char): The Telephony Layer for AI Voice Agents (2026 Guide)

Meta description: Connect Vapi, Retell, ElevenLabs, and a dozen more AI voice platforms to phone numbers. SIP trunking, WebSocket, and BYOC explained, with setup steps for each.

Target queries: how to connect Vapi to a phone number, how to add a phone number to Retell AI, ElevenLabs SIP trunking, LiveKit SIP telephony, Pipecat phone number, BYOC for AI voice agents, SIP trunk for AI voice agent

Platform "how to connect X" queries (highest commercial intent, no Vobiz mention needed):

• How to connect Vapi / Retell AI / ElevenLabs / LiveKit / Pipecat to a phone number
• Same pattern for Bolna.ai, Sarvam Samvaad, OpenAI Realtime, Gemini Live, Deepgram, Ultravox, Smallest AI, Dograh, Rapida AI, xAI, Agora

Definitional queries (people who don't have the vocabulary yet):

• What is a SIP trunk / the telephony layer for AI voice agents
• SIP trunk vs. phone number
• SIP trunking vs. WebSocket
• What is BYOC, and BYOC vs. number porting

Evaluation-stage queries (closest to a buying decision):

• Best SIP trunk provider for AI voice agents
• How much latency does SIP trunking add
• Is SIP trunking secure for healthcare/finance
• Can I use the same number for inbound and outbound AI calls

Why it should get cited, not just indexed:

• FAQ H3s phrased as real queries, answer-first — the shape AI Overviews/Perplexity pull from
• Two comparison tables get lifted as structured data
• Every Vobiz claim ties to a number (sub-80ms, 130+ countries) or named source (Gartner, Market.us)
• Dense named-entity coverage (Deepgram, GPT-4o, ElevenLabs, SRTP, TLS, PSTN, DID) that models can cite confidently