> ## Documentation Index
> Fetch the complete documentation index at: https://www.vobiz.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deepgram Voice Agent

> Bridge a Vobiz XML <Stream> to the Deepgram Voice Agent API for a phone-callable agent with Flux STT, Flux TTS, real barge-in, and playback confirmation.

<img className="block w-14 h-14 rounded-xl border border-gray-200 dark:border-zinc-800 mb-4" src="https://mintcdn.com/vobizai/TSi6bV2yJ4DOAsqc/images/deepgram/logo.png?fit=max&auto=format&n=TSi6bV2yJ4DOAsqc&q=85&s=93c783aab94de4da08b0a8b10c380468" alt="Deepgram" width="460" height="460" data-path="images/deepgram/logo.png" />

The [Deepgram Voice Agent API](https://developers.deepgram.com/docs/voice-agent) runs speech-to-text, the LLM and text-to-speech behind a single WebSocket, with its own turn-taking. Bridge it to a Vobiz [`<Stream>`](/docs/xml/stream) and you have a phone-callable agent that a caller can interrupt mid-sentence — no separate STT vendor, no TTS vendor, and no turn logic of your own.

**Source code:** [vobiz-ai/Vobiz-Deepgram-Voice-Agent](https://github.com/vobiz-ai/Vobiz-Deepgram-Voice-Agent) — the reference FastAPI bridge used throughout this guide (`app.py`, `call.py`, `mock_vobiz.py`), plus a mock client that exercises the whole protocol without placing a call.

<Note>
  **Scope:** inbound and outbound. Attach a number to an application whose answer URL points at `/answer`, or place an outbound call with the same URL — the bridge does not care which direction the call came from.
</Note>

## How it works

<div className="my-6">
  <img className="block dark:hidden w-full max-w-2xl mx-auto" src="https://mintcdn.com/vobizai/d1twzJfy97_OnWYw/images/deepgram/how-it-works-light.svg?fit=max&auto=format&n=d1twzJfy97_OnWYw&q=85&s=1a35f654c8efecd2294e8316c5eeb680" alt="Vertical call flow: caller audio travels from the caller over PSTN to Vobiz, over a bidirectional WebSocket as media events to app.py, and into the Deepgram Voice Agent as send_media. Agent audio returns as audio bytes, then playAudio, then PSTN. A dashed barge-in loop runs from Deepgram back to Vobiz, labelled UserStartedSpeaking then clearAudio." width="720" height="500" data-path="images/deepgram/how-it-works-light.svg" />

  <img className="hidden dark:block w-full max-w-2xl mx-auto" src="https://mintcdn.com/vobizai/d1twzJfy97_OnWYw/images/deepgram/how-it-works-dark.svg?fit=max&auto=format&n=d1twzJfy97_OnWYw&q=85&s=f107224b6a9621911773b137c2bb5019" alt="Vertical call flow: caller audio travels from the caller over PSTN to Vobiz, over a bidirectional WebSocket as media events to app.py, and into the Deepgram Voice Agent as send_media. Agent audio returns as audio bytes, then playAudio, then PSTN. A dashed barge-in loop runs from Deepgram back to Vobiz, labelled UserStartedSpeaking then clearAudio." width="720" height="500" data-path="images/deepgram/how-it-works-dark.svg" />
</div>

Vobiz opens the WebSocket **to you** — your `wss://` URL is the server. Caller audio arrives as `media` events, the agent's speech goes back as `playAudio`, and `app.py` is a thin relay: format handling, barge-in, and the Vobiz control protocol. Deepgram owns the conversation.

1. Vobiz fetches `/answer` and receives a `<Stream>` element.
2. Vobiz opens a WebSocket to `/media/<secret>` and sends a `start` event.
3. Caller audio arrives as `media` events and is forwarded to Deepgram.
4. Agent audio comes back as raw bytes and goes to Vobiz as `playAudio` events.
5. When the caller interrupts, Deepgram signals it and the app sends `clearAudio`.
6. After each turn a `checkpoint` is sent; Vobiz replies `playedStream` once the caller has actually heard it.

<Card title="The bidirectional Stream protocol" icon="wave-square" href="/docs/xml/stream/stream-events" horizontal>
  Every event and control message on the socket — `start`, `media`, `dtmf`, `playedStream`, `clearedAudio`, `stop` — and what you send back.
</Card>

## Audio profiles

The two directions of a Vobiz bidirectional stream are configured **independently**. Both profiles below are pure passthrough — the bridge never resamples.

| `AUDIO_MODE`        | Vobiz → app (`<Stream contentType>`) | app → Vobiz (`playAudio`) | Deepgram agent settings                |
| ------------------- | ------------------------------------ | ------------------------- | -------------------------------------- |
| `mulaw` *(default)* | `audio/x-mulaw;rate=8000`            | mu-law 8000               | `mulaw` 8k in / `mulaw` 8k out         |
| `l16`               | `audio/x-l16;rate=16000`             | L16 24000                 | `linear16` 16k in / `linear16` 24k out |

`mulaw` matches the PSTN leg exactly and is the right default. `l16` trades bandwidth for a wider band — 16 kHz in, 24 kHz out — and Deepgram emits 24 kHz natively, so nothing is degraded on the way to Vobiz.

<Warning>
  **Never put `rate=24000` on `<Stream contentType>`.** That attribute configures the **inbound** direction, which tops out at 16 kHz. 24 kHz is outbound-only — valid on `playAudio` and nowhere else.
</Warning>

`playAudio` accepts L16 at 8/16/24 kHz and mu-law at 8 kHz. See [audio formats](/docs/xml/stream/audio-formats) for the full matrix.

## Requirements

| Requirement                 | Detail                                                                                   |
| --------------------------- | ---------------------------------------------------------------------------------------- |
| Deepgram API key            | From [console.deepgram.com](https://console.deepgram.com) — it runs STT, the LLM and TTS |
| Vobiz account               | `AUTH_ID`, `AUTH_TOKEN`, and a DID — [console.vobiz.ai](https://console.vobiz.ai)        |
| Public HTTPS + WSS endpoint | Vobiz connects inbound to your `wss://` URL. ngrok is fine in development.               |
| Runtime                     | Python 3.11+                                                                             |

<Info>
  No separate LLM key is needed. Deepgram manages the OpenAI connection and bills it through your Deepgram account.
</Info>

<Info>
  If your WebSocket endpoint is IP-restricted, allow **inbound TCP 443** from the Vobiz media fleet. The RTP rule (UDP 5000–65535) does not cover it — see [IP whitelisting](/docs/concepts/ip-whitelisting#websocket-streaming).
</Info>

## Step 1: Configure the bridge

```bash theme={null}
git clone https://github.com/vobiz-ai/Vobiz-Deepgram-Voice-Agent
cd Vobiz-Deepgram-Voice-Agent
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env       # then fill it in
```

```ini .env theme={null}
# ---- Required ----
DEEPGRAM_API_KEY=
# Public host Vobiz reaches, no scheme.
PUBLIC_HOSTNAME=your-host.example.com

# ---- Optional ----
HTTP_PORT=5050
# mulaw (default) or l16 — see Audio profiles above.
AUDIO_MODE=mulaw

# ---- Security ----
# Alphanumeric — it becomes a path segment on the wss:// URL.
#   python -c "import secrets; print(secrets.token_hex(16))"
STREAM_SECRET=
# true to validate the X-Vobiz-Signature-V3 HMAC on /answer.
VERIFY_SIGNATURE=

# ---- Outbound only (used by call.py) ----
VOBIZ_AUTH_ID=
VOBIZ_AUTH_TOKEN=
FROM_NUMBER=+91XXXXXXXXXX
TO_NUMBER=+91XXXXXXXXXX
```

### Dependencies

```text requirements.txt theme={null}
fastapi
uvicorn[standard]
deepgram-sdk>=7.7.0
python-dotenv
python-multipart
websockets
requests
```

## Step 2: Run it

```bash theme={null}
python app.py
```

Expose the server publicly, then confirm what it resolved:

```bash theme={null}
curl -s https://<public>/health | python -m json.tool
```

```json theme={null}
{
  "answer_url": "https://<public>/answer",
  "stream_url": "wss://<public>/media/<secret>",
  "audio_mode": "mulaw",
  "vobiz_to_app": "audio/x-mulaw;rate=8000",
  "app_to_vobiz": "audio/x-mulaw;rate=8000",
  "play_frame_bytes": 160,
  "agent": { "listen": "flux-general-en", "think": "gpt-4o-mini", "speak": "flux-alexis-en" },
  "answer_signature_checked": false,
  "stream_secret_checked": true
}
```

Whichever host you use becomes the answer URL below.

| Route                 | Role                                                                       |
| --------------------- | -------------------------------------------------------------------------- |
| `GET/POST /answer`    | The XML Vobiz executes when the call is answered                           |
| `WS /media/<secret>`  | The bidirectional media stream `<Stream>` connects to                      |
| `POST /stream-status` | `<Stream statusCallbackUrl>` — `StartStream`, `PlayedStream`, `StopStream` |
| `POST /hangup`        | The call's `hangup_url`                                                    |
| `GET /health`         | Resolved URLs and audio profile                                            |

## Step 3: Place a call

<Tabs>
  <Tab title="Inbound">
    Vobiz decides what to do with an inbound call by looking up the **Voice Application** attached to the number that was dialled. A number on its own is not enough — create the application first, then attach a number to it.

    **1. Create a Voice Application**

    In the console, go to **Voice Applications → Create application**. Set **Primary answer URL** to `https://<public>/answer` with method **POST**. Optionally set the **Hangup URL** to `https://<public>/hangup` to receive the call-ended webhook.

    <Frame>
      <img src="https://mintcdn.com/vobizai/TSi6bV2yJ4DOAsqc/images/deepgram/create-voice-application.png?fit=max&auto=format&n=TSi6bV2yJ4DOAsqc&q=85&s=431eea4c0756b1bd326d2285925b401a" alt="Vobiz console Create application dialog with fields for application name, primary answer URL with a POST method selector, hangup URL, and fallback answer URL" width="2000" height="1416" data-path="images/deepgram/create-voice-application.png" />
    </Frame>

    **2. Attach a number**

    Open the application and attach one of your DIDs under **Attached Numbers → Attach number**. Calls to that number now fetch XML from your answer URL.

    <Frame>
      <img src="https://mintcdn.com/vobizai/TSi6bV2yJ4DOAsqc/images/deepgram/attach-number.png?fit=max&auto=format&n=TSi6bV2yJ4DOAsqc&q=85&s=5fbf399f605e1a903a27e7dbff89f44b" alt="Attached Numbers panel on a Vobiz voice application showing no numbers attached yet and an Attach number button" width="980" height="376" data-path="images/deepgram/attach-number.png" />
    </Frame>

    Dial the attached number with a `0` or `+91` prefix — `09XXXXXXXXX` or `+919XXXXXXXXX`. You should hear the greeting.

    See [Applications](/docs/applications) for the full reference.
  </Tab>

  <Tab title="Outbound">
    `call.py` places the call and points it at this server, so no Voice Application is needed:

    ```bash theme={null}
    python call.py --to +919XXXXXXXXX
    ```

    It posts to [Make a Call](/docs/call/make-call):

    ```bash theme={null}
    curl -X POST "https://api.vobiz.ai/api/v1/Account/$AUTH_ID/Call/" \
      -H "X-Auth-ID: $AUTH_ID" \
      -H "X-Auth-Token: $AUTH_TOKEN" \
      -H "Content-Type: application/json" \
      -d '{
        "from": "+91XXXXXXXXXX",
        "to": "+91XXXXXXXXXX",
        "answer_url": "https://<public>/answer",
        "answer_method": "POST",
        "hangup_url": "https://<public>/hangup",
        "hangup_method": "POST"
      }'
    ```

    Answer the phone and talk. `app.py` prints `[user]` and `[assistant]` lines as the conversation runs.
  </Tab>
</Tabs>

## The answer XML

```xml theme={null}
<?xml version="1.0" encoding="UTF-8"?>
<Response>
  <Stream bidirectional="true"
          audioTrack="inbound"
          keepCallAlive="true"
          contentType="audio/x-mulaw;rate=8000"
          statusCallbackUrl="https://example.com/stream-status"
          statusCallbackMethod="POST">wss://example.com/media/&lt;secret&gt;</Stream>
  <Hangup/>
</Response>
```

| Attribute              | Why it matters                                                                                                                                               |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `bidirectional="true"` | Without it the socket is receive-only and `playAudio` is ignored. It is also what unlocks `clearAudio` and `checkpoint`                                      |
| `keepCallAlive="true"` | Holds XML execution while the socket is open. Without it the element returns immediately, the document ends, and Vobiz hangs up on *End Of XML Instructions* |
| `audioTrack="inbound"` | The only track setting valid alongside `bidirectional` — `both` is rejected. It also stops the agent hearing its own playback                                |
| `contentType`          | Controls **Vobiz → app** only. Never the playback format                                                                                                     |
| `statusCallbackUrl`    | HTTP mirror of the socket lifecycle: `StartStream`, `PlayedStream`, `ClearedAudio`, `DroppedStream`, `StopStream`                                            |

The WebSocket URL is the element's **text content**, not a `url=""` attribute.

<Warning>
  **Malformed answer XML is not an HTTP error.** Vobiz accepts the `200`, then drops the call about a second later, and the only trace is the CDR field `hangup_cause_name: "Invalid Answer XML"`. A URL carrying two query parameters contains a bare `&`, which is enough on its own to invalidate the document — the reference bridge runs every interpolated value through `html.escape()` for this reason.
</Warning>

## Inside the bridge

### The agent settings

Both ends of the pipeline are [Flux](https://developers.deepgram.com/docs/flux/feature-overview), Deepgram's conversational speech models, and both live on v2 endpoints — so each provider must pin `version: "v2"`. Omit it and the provider falls back to v1, where the Flux model names are not valid.

```python theme={null}
AGENT_SETTINGS = {
    "type": "Settings",
    "audio": {"input": PROFILE["agent_input"], "output": PROFILE["agent_output"]},
    "agent": {
        "language": "en",
        "listen": {"provider": {"type": "deepgram", "version": "v2", "model": "flux-general-en"}},
        "think":  {"provider": {"type": "open_ai", "model": "gpt-4o-mini"}, "prompt": PROMPT},
        "speak":  {"provider": {"type": "deepgram", "version": "v2", "model": "flux-alexis-en"}},
        "greeting": GREETING,
    },
}
```

Sending the settings frame is what starts the conversation, greeting included. To swap the LLM, change the `think` provider — for example `{"type": "anthropic", "model": "claude-sonnet-5"}`.

### Barge-in

Deepgram detects the caller talking over the agent and emits `UserStartedSpeaking`. Vobiz may still have seconds of the agent's reply buffered, so the bridge flushes it:

```python theme={null}
if isinstance(message, AgentV1UserStartedSpeaking):
    await stream.clear()   # {"event": "clearAudio", "streamId": ...}
```

Vobiz confirms with a `clearedAudio` event. Without this the agent keeps talking over the caller. See [`clearAudio`](/docs/xml/stream/clear-audio).

### Knowing the caller actually heard it

`playAudio` means *sent*, not *heard*. After each turn the bridge sends a [`checkpoint`](/docs/xml/stream/checkpoint-event) and Vobiz answers `playedStream` once the buffered audio has played out:

```python theme={null}
if isinstance(message, AgentV1AgentAudioDone):
    await stream.checkpoint()   # {"event": "checkpoint", "name": "turn-3", ...}
```

That signal is what lets an agent say goodbye and then hang up without clipping its own last word.

### Frame slicing

Deepgram hands over arbitrarily sized audio chunks. Vobiz is happiest with steady telephony-sized frames, so `VobizStream.play()` re-slices into 20 ms — **160 bytes** mu-law at 8 kHz, **960 bytes** L16 at 24 kHz — rather than forwarding blindly.

### Format mismatches are silent

Since the bridge never resamples, a `contentType` that disagrees with `AUDIO_MODE` is just garbage audio into the agent. `read_start()` compares the XML against `start.mediaFormat` and prints `[audio] WARNING` instead of failing quietly.

## Security

Both public endpoints are reachable by anyone who learns the URL, so the bridge ships two opt-in controls:

| Control            | What it protects | How                                                                                                                                         |
| ------------------ | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `STREAM_SECRET`    | The media socket | A random alphanumeric string becomes a path segment on the `wss://` URL and is checked on connect. A mismatch closes the socket with `1008` |
| `VERIFY_SIGNATURE` | `/answer`        | Validates the `X-Vobiz-Signature-V3` (or V2) HMAC, keyed by `VOBIZ_AUTH_TOKEN`                                                              |

<Note>
  **`extraHeaders` cannot authenticate the media socket.** The values never reach the WebSocket — not as an upgrade header, and not in the `start` frame, whose `extra_headers` field stays the literal `"{}"`. They surface only in the `statusCallbackUrl` payload, as `X-VH-<key>`. That makes `extraHeaders` status-callback metadata rather than stream credentials, which is why the secret rides in the URL path instead.
</Note>

<Note>
  **The webhook signature covers the URL and a nonce, never the body.** Voice webhooks are form-encoded, so any scheme that hashes a JSON body will not verify. Query parameters are stripped first, and behind a tunnel the public URL has to be rebuilt — `request.url` is the internal address Vobiz never saw. Signature headers are only emitted when the callback URL has auth credentials configured on it, which is why `VERIFY_SIGNATURE` is opt-in. See [Validating callbacks](/docs/concepts/validating-callbacks).
</Note>

## Test without placing a call

`mock_vobiz.py` stands in for Vobiz — it speaks the media-stream protocol against a running `app.py`, answers `checkpoint` with `playedStream`, and reports what came back.

```bash theme={null}
python mock_vobiz.py                        # 4s of silence — transport and greeting
python mock_vobiz.py --wav question.wav     # stream a real question (mono, profile sample rate)
```

A pass means `playAudio` frames came back in the format the XML asked for.

## Configuration reference

| Variable           | Required | Default  | Notes                                                  |
| ------------------ | -------- | -------- | ------------------------------------------------------ |
| `DEEPGRAM_API_KEY` | Yes      | —        | Runs the whole agent — STT, LLM and TTS                |
| `PUBLIC_HOSTNAME`  | Yes      | —        | Public host Vobiz reaches, **no scheme**               |
| `HTTP_PORT`        | No       | `5050`   | Server port                                            |
| `AUDIO_MODE`       | No       | `mulaw`  | `mulaw` or `l16`                                       |
| `STREAM_SECRET`    | No       | *(none)* | Alphanumeric; becomes a path segment on the stream URL |
| `VERIFY_SIGNATURE` | No       | *(off)*  | `true` to validate the HMAC on `/answer`               |
| `VOBIZ_AUTH_ID`    | Outbound | —        | Only needed by `call.py`                               |
| `VOBIZ_AUTH_TOKEN` | Outbound | —        | Also the webhook signing key                           |
| `FROM_NUMBER`      | Outbound | —        | A DID this account owns                                |
| `TO_NUMBER`        | Outbound | —        | Default destination for `call.py`                      |

## Troubleshooting

| Symptom                                                        | Cause                                                                                                             | Resolution                                                      |
| -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| Call hangs up immediately; log shows *End Of XML Instructions* | `keepCallAlive="true"` missing, or `audioTrack="both"` with `bidirectional="true"`                                | Set `keepCallAlive="true"` and `audioTrack="inbound"`           |
| Silence in both directions                                     | `bidirectional="true"` missing — `playAudio` is ignored on a one-way stream                                       | Add the attribute                                               |
| Garbled or chipmunk audio                                      | `<Stream contentType>` and `AUDIO_MODE` disagree                                                                  | Match them; the app prints `[audio] WARNING` on the start event |
| Agent talks over the caller                                    | `clearAudio` is not reaching Vobiz                                                                                | Check `streamId` is set before the first `playAudio`            |
| Stream connects but no usable media arrives                    | `rate=24000` set on `<Stream contentType>`                                                                        | 24 kHz is outbound-only — use 8000 or 16000 inbound             |
| WebSocket closes with `1008`                                   | `STREAM_SECRET` does not match the secret in the stream URL path                                                  | Re-copy the value, or clear it while testing                    |
| `/answer` returns `403`                                        | `VERIFY_SIGNATURE=true` but the callback URL has no auth credentials configured, so no signature headers are sent | Set the credentials in the console first, then enable it        |
| Outbound call returns `401` or `402`                           | `401` credentials, `402` balance. *"from number … not owned"* means the DID belongs to another account            | Check `VOBIZ_AUTH_ID`/`VOBIZ_AUTH_TOKEN` and the `from` DID     |
| Inbound call is never answered                                 | The number has no Voice Application attached, or the application's answer URL does not point at this server       | See [Step 3 → Inbound](#step-3-place-a-call)                    |
| Inbound call does not connect                                  | The number was dialled without a prefix                                                                           | Dial with `0` or `+91`                                          |
| Hangup webhook reports zero cost                               | Voice and stream are billed as separate line items that appear only in the CDR                                    | Read [`GET /cdr/{call_uuid}`](/docs/cdr/get-cdr)                     |

## Next steps

* Clone the reference bridge: [vobiz-ai/Vobiz-Deepgram-Voice-Agent](https://github.com/vobiz-ai/Vobiz-Deepgram-Voice-Agent)
* Read the [`<Stream>` reference](/docs/xml/stream), [audio formats](/docs/xml/stream/audio-formats), [stream events](/docs/xml/stream/stream-events) and [`playAudio`](/docs/xml/stream/play-audio)
* Deepgram's [Voice Agent API](https://developers.deepgram.com/docs/voice-agent) and [Flux](https://developers.deepgram.com/docs/flux/feature-overview) docs
* Bridging a different model over the same socket? See [WebSockets](/docs/integrations/websockets) and [Gemini Live](/docs/integrations/gemini-live)