Pipecat integration

This integration combines Vobiz telephony with the Pipecat voice agent framework to build intelligent AI-powered phone calls.

Overview

What you'll build: An outbound calling system with real-time AI conversations powered by OpenAI (STT → LLM → TTS), automatic call recording, and bidirectional audio streaming.

Features

• AI Voice Conversations: Natural conversations powered by OpenAI GPT + TTS/STT

• Outbound Calling: Trigger calls via REST API from anywhere

• Automatic Recording: All conversations automatically recorded and saved

• Real-time Streaming: Bidirectional audio via WebSockets

Prerequisites

Installation

Clone the repository

bashCopy
git clone https://github.com/vobiz-ai/Vobiz-X-Pipecat
cd Vobiz-X-Pipecat

Install dependencies

bashCopy
pip install -r requirements.txt

ℹ️ Note: This installs FastAPI, Pipecat, OpenAI SDK, and other required packages.

Configure environment

The repo ships an env.example file. Copy it and fill in your values:

bashCopy
cp env.example .env

bashCopy
# Required
OPENAI_API_KEY=sk-...
VOBIZ_AUTH_ID=MA_XXXXXXXX
VOBIZ_AUTH_TOKEN=your-token-here
PUBLIC_URL=https://your-ngrok-url.ngrok-free.app

# Optional
VOBIZ_PHONE_NUMBER=+918065480214        # default `from` for the /start endpoint
DEEPGRAM_API_KEY=...                    # use Deepgram for STT instead of OpenAI

Where to find these values:

• OPENAI_API_KEY - OpenAI Platform → API Keys
• VOBIZ_AUTH_ID - Vobiz Console → Account Settings
• VOBIZ_AUTH_TOKEN - Vobiz Console → Account Settings
• PUBLIC_URL - Your ngrok URL (set in Step 2 of Usage)
• VOBIZ_PHONE_NUMBER (optional) - Vobiz number used as the caller-ID for /start
• DEEPGRAM_API_KEY (optional) - drop-in alternative to OpenAI Whisper for STT

Usage

Start the server

bashCopy
python server.py

The server runs on http://0.0.0.0:7860.

Start ngrok

In a new terminal, expose your local server:

bashCopy
ngrok http 7860

Copy the ngrok URL from the output (for example, https://abc123.ngrok-free.app).

⚠️ Warning: Important: Update PUBLIC_URL in your .env file with this ngrok URL, then restart the server.

Make a call

There are two ways to trigger an outbound call - pick whichever fits your stack.

The repo exposes `POST /start` on the local server. It wraps the Vobiz Call API and auto-fills `answer_url` from your `PUBLIC_URL`, plus uses `VOBIZ_PHONE_NUMBER` as `from` when set.

```bash
curl -X POST http://localhost:7860/start \
  -H "Content-Type: application/json" \
  -d '{ "to": "+919148223344" }'
```

Direct Vobiz API

bashCopy
curl -X POST https://api.vobiz.ai/api/v1/Account/YOUR_AUTH_ID/Call/ \
  -H "X-Auth-ID: YOUR_AUTH_ID" \
  -H "X-Auth-Token: YOUR_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "from": "+918011223344",
    "to": "+919148223344",
    "answer_url": "https://your-ngrok-url.ngrok-free.app/answer",
    "answer_method": "POST"
  }'

What happens next:

• Phone rings at the "to" number
• When answered, Vobiz requests XML from your server's /answer endpoint
• Server returns a WebSocket URL pointing at /ws
• Audio streams bidirectionally to the Pipecat bot
• AI assistant speaks and listens (STT → LLM → TTS)
• Conversation is automatically recorded; MP3 fetched via /recording-ready

Receiving inbound calls

Configure your Vobiz number to handle incoming calls with your Pipecat agent.

Open Applications

Log in to the Vobiz Console and navigate to the Applications section in the sidebar.

Create an application

Click Create New Application and give it a name (for example, "Pipecat Agent").

Configure URLs

Set the Answer URL to your ngrok URL (for example, https://.../answer) and select POST method. You can use the same URL for Hangup or leave it blank.

Assign phone number

Go to Phone Numbers, select your number, and assign it to the application you just created.

✅ Success: Success: Calls to your Vobiz number will now be handled by your local Pipecat server!

Quick reference

Server endpoints

Project files

Customizing the bot

Edit bot.py to customize your AI assistant:

pythonCopy
messages = [
    {
        "role": "system",
        "content": "You are a friendly customer service agent..."
    },
]

pythonCopy
tts = OpenAITTSService(
    api_key=os.getenv("OPENAI_API_KEY"),
    voice="nova",  # Options: alloy, echo, fable, onyx, nova, shimmer
)

✅ Success: Integration complete!

You can now make AI-powered phone calls with Vobiz and Pipecat.

Next steps

• Customize your AI assistant's personality in bot.py
• Deploy to production (AWS/GCP/Heroku) instead of ngrok
• Add custom business logic and integrations

Resources

Vobiz Documentation

• API Documentation
• Make a Call
• Vobiz Console

External Resources

• GitHub Repository
• Pipecat Documentation

Build it with an AI agent

<Prompt description="Clone, configure, and run the Vobiz-X-Pipecat repo - your first AI voice agent in ~5 minutes." icon="robot" actions={["copy", "cursor"]}

Using the Vobiz MCP at https://vobiz.ai/docs/mcp:

1. Read the vobiz-ai-voice-agents skill from mintlify://skills/ai-voice-agents and the vobiz-audio-streams skill from mintlify://skills/audio-streams.
2. Clone https://github.com/vobiz-ai/Vobiz-X-Pipecat into the current directory.
3. Run pip install -r requirements.txt (warn me first if I'm not in a virtualenv).
4. Copy env.example to .env and ask me for: OPENAI_API_KEY, VOBIZ_AUTH_ID, VOBIZ_AUTH_TOKEN, my Vobiz number for VOBIZ_PHONE_NUMBER. Leave PUBLIC_URL and DEEPGRAM_API_KEY blank for now.
5. Explain how to start the FastAPI server (python server.py) and how to expose it with ngrok http 7860. Tell me to paste the ngrok HTTPS URL into PUBLIC_URL and restart.
6. Show me the curl to fire my first AI-powered call via the local /start endpoint: POST http://localhost:7860/start with body {"to": "+91..."}.
7. Explain the call flow: Vobiz → /answer → returns XML pointing at /ws → bidirectional audio → Pipecat pipeline (STT → LLM → TTS) → MP3 recording delivered to /recording-ready.