Pipecat integration

This integration combines Vobiz telephony with the Pipecat voice agent framework to build intelligent AI-powered phone calls.

Overview

What you’ll build: An outbound calling system with real-time AI conversations powered by OpenAI (STT → LLM → TTS), automatic call recording, and bidirectional audio streaming.

Call flow:

curl POST

→

Vobiz API

→

Call initiated

→

Call answered

→

Vobiz requests XML

→

Server returns WebSocket URL

→

Audio streams

→

Pipecat bot (STT → LLM → TTS)

→

AI conversation + Recording

Features

AI Voice Conversations

Natural conversations powered by OpenAI GPT + TTS/STT

Outbound Calling

Trigger calls via REST API from anywhere

Automatic Recording

All conversations automatically recorded and saved

Real-time Streaming

Bidirectional audio via WebSockets

Prerequisites

Vobiz Account with Auth ID and Auth Token → Sign up

OpenAI API Key for LLM, STT, and TTS → Get API key

Python 3.10+ installed on your system

ngrok for local development → Download ngrok

Installation

Clone the repository

git clone https://github.com/vobiz-ai/Vobiz-X-Pipecat
cd Vobiz-X-Pipecat

Install dependencies

pip install -r requirements.txt

This installs FastAPI, Pipecat, OpenAI SDK, and other required packages.

Configure environment

The repo ships an env.example file. Copy it and fill in your values:

cp env.example .env

.env

# Required
OPENAI_API_KEY=sk-...
VOBIZ_AUTH_ID=MA_XXXXXXXX
VOBIZ_AUTH_TOKEN=your-token-here
PUBLIC_URL=https://your-ngrok-url.ngrok-free.app

# Optional
VOBIZ_PHONE_NUMBER=+918065480214        # default `from` for the /start endpoint
DEEPGRAM_API_KEY=...                    # use Deepgram for STT instead of OpenAI

Where to find these values:

OPENAI_API_KEY - OpenAI Platform → API Keys
VOBIZ_AUTH_ID - Vobiz Console → Account Settings
VOBIZ_AUTH_TOKEN - Vobiz Console → Account Settings
PUBLIC_URL - Your ngrok URL (set in Step 2 of Usage)
VOBIZ_PHONE_NUMBER (optional) - Vobiz number used as the caller-ID for /start
DEEPGRAM_API_KEY (optional) - drop-in alternative to OpenAI Whisper for STT

Usage

Start the server

python server.py

The server runs on http://0.0.0.0:7860.

Start ngrok

In a new terminal, expose your local server:

ngrok http 7860

Copy the ngrok URL from the output (for example, https://abc123.ngrok-free.app).

Important: Update PUBLIC_URL in your .env file with this ngrok URL, then restart the server.

Make a call

There are two ways to trigger an outbound call - pick whichever fits your stack.

Server's /start helper (easiest)
Direct Vobiz API

The repo exposes POST /start on the local server. It wraps the Vobiz Call API and auto-fills answer_url from your PUBLIC_URL, plus uses VOBIZ_PHONE_NUMBER as from when set.

curl -X POST http://localhost:7860/start \
  -H "Content-Type: application/json" \
  -d '{ "to": "+919148223344" }'

curl -X POST https://api.vobiz.ai/api/v1/Account/YOUR_AUTH_ID/Call/ \
  -H "X-Auth-ID: YOUR_AUTH_ID" \
  -H "X-Auth-Token: YOUR_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "from": "+918011223344",
    "to": "+919148223344",
    "answer_url": "https://your-ngrok-url.ngrok-free.app/answer",
    "answer_method": "POST"
  }'

What happens next:

Phone rings at the “to” number
When answered, Vobiz requests XML from your server’s /answer endpoint
Server returns a WebSocket URL pointing at /ws
Audio streams bidirectionally to the Pipecat bot
AI assistant speaks and listens (STT → LLM → TTS)
Conversation is automatically recorded; MP3 fetched via /recording-ready

Receiving inbound calls

Configure your Vobiz number to handle incoming calls with your Pipecat agent.

Open Applications

Create an application

Click Create New Application and give it a name (for example, “Pipecat Agent”).

Configure URLs

Set the Answer URL to your ngrok URL (for example, https://.../answer) and select POST method. You can use the same URL for Hangup or leave it blank.

Assign phone number

Go to Phone Numbers, select your number, and assign it to the application you just created.

Success: Calls to your Vobiz number will now be handled by your local Pipecat server!

Quick reference

Server endpoints

Endpoint	Method	Description
`/start`	POST	Trigger an outbound call. Body: `{ "to": "+91..." }`. Wraps the Vobiz Call API.
`/answer`	POST	Called by Vobiz when the call is answered. Returns XML pointing at `/ws`.
`/ws`	WebSocket	Bidirectional audio between Vobiz and the Pipecat bot.
`/recording-finished`	POST	Vobiz callback when recording stops. Logs metadata.
`/recording-ready`	POST	Vobiz callback when the recording is ready. Auto-downloads the MP3 via `download_recording.py`.

Project files

File	Purpose
`server.py`	FastAPI app - owns `/start`, `/answer`, `/ws`, and the recording callbacks
`bot.py`	Pipecat pipeline - STT → LLM → TTS, runs per call
`download_recording.py`	Helper used by `/recording-ready` to pull the MP3 from Vobiz
`requirements.txt`	Python dependencies (FastAPI, Pipecat, OpenAI SDK, etc.)
`env.example`	Template for your `.env` file - copy and fill in

Customizing the bot

Edit bot.py to customize your AI assistant:

Change Bot Personality

messages = [
    {
        "role": "system",
        "content": "You are a friendly customer service agent..."
    },
]

Change TTS Voice

tts = OpenAITTSService(
    api_key=os.getenv("OPENAI_API_KEY"),
    voice="nova",  # Options: alloy, echo, fable, onyx, nova, shimmer
)

Integration complete!You can now make AI-powered phone calls with Vobiz and Pipecat.

Next steps

Customize your AI assistant’s personality in bot.py
Deploy to production (AWS/GCP/Heroku) instead of ngrok
Add custom business logic and integrations

Resources

Vobiz Documentation

External Resources

Build it with an AI agent

Clone, configure, and run the Vobiz-X-Pipecat repo - your first AI voice agent in ~5 minutes.

Open in Cursor

AI Voice Platforms

Official SDKs

Browser & WebRTC

Business Phone Systems

Migrations

Pipecat integration

Overview

Features

AI Voice Conversations

Outbound Calling

Automatic Recording

Real-time Streaming

Prerequisites

Installation

Usage

Receiving inbound calls

Quick reference

Server endpoints

Project files

Customizing the bot

Next steps

Resources

Build it with an AI agent

​Overview

​Features

AI Voice Conversations

Outbound Calling

Automatic Recording

Real-time Streaming

​Prerequisites

​Installation

​Usage

​Receiving inbound calls

​Quick reference

​Server endpoints

​Project files

​Customizing the bot

​Next steps

​Resources

​Build it with an AI agent

Overview

Features

Prerequisites

Installation

Usage

Receiving inbound calls

Quick reference

Server endpoints

Project files

Customizing the bot

Next steps

Resources

Build it with an AI agent