OpenAI Realtime SIP Call Handler

Connect inbound calls on a Vobiz number directly to OpenAI's Realtime API. The AI answers the call and holds a natural, low-latency conversation with the caller.

ℹ️ Note: Scope: inbound calls only. A caller dials a Vobiz number and is answered by the AI agent.

Source code: vobiz-ai/Vobiz-Openai-Realtime - the reference Flask service used throughout this guide (app.py, requirements.txt, .env.example).

How it works

Vobiz does not proxy the audio. The SIP INVITE is handed to OpenAI, which terminates the media directly with the caller. Your application server only performs control-plane work over HTTPS and a WebSocket - it never handles RTP.

textCopy
Caller
  └─> Vobiz DID
        └─> Inbound Trunk  (transport: TLS, secure: true)
              └─> Origination URI  →  sip:{PROJECT_ID}@sip.api.openai.com:5061
                    └─> OpenAI fires  realtime.call.incoming  webhook
                          └─> Your server: POST /v1/realtime/calls/{call_id}/accept
                                └─> Your server: wss://api.openai.com/v1/realtime?call_id=…
                                      └─> SRTP media flows OpenAI ⟷ caller

Because OpenAI terminates the media, VobizXML verbs such as <Stream> are not used in this integration, and audio resampling is not required.

Requirements

Network requirements

OpenAI publishes the following media CIDR blocks. Any firewall between Vobiz and OpenAI must permit bidirectional UDP with them:

textCopy
13.79.45.80/28    23.98.140.64/28    40.67.149.176/28    40.83.204.240/28

Regional endpoints

Step 1: Create the origination URI

Console: SIP → Inbound → Origination URIs (console.vobiz.ai/app/sip/in/uri)

⚠️ Warning: The port :5061 is mandatory.

OpenAI accepts SIP only on 5061. Without an explicit port the URI resolves to the SIP default (5060), where OpenAI is not listening. The call is refused back to the carrier before it reaches OpenAI, and the CDR shows hangup_disposition: send_refuse with terminated_to: inbound_carrier.

Substitute your own Project ID, found at platform.openai.com → Settings → Project → General.

API equivalent

bashCopy
curl -X POST https://api.vobiz.ai/api/v1/Account/$AUTH_ID/origination-uris \
-H "X-Auth-ID: $AUTH_ID" \
-H "X-Auth-Token: $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "openai-realtime",
"sip_uri": "sip:proj_xxxxxxxx@sip.api.openai.com:5061",
"priority": 1
}'

The response returns the new URI's id, which you pass as primary_uri_uuid when creating the trunk. transport is not settable on Create Origination URI - select TLS in the console.

Step 2: Create the inbound trunk

Console: SIP → Inbound → Trunks (console.vobiz.ai/app/sip/in/trunks)

⚠️ Warning: secure must be enabled, and it is distinct from transport.

• transport controls signalling encryption (TLS).
• secure controls media encryption (SRTP).

OpenAI requires SRTP. If transport is TLS but secure is left disabled, the call completes signalling, answers, and bills normally - but no audio passes. The symptom is a connected, billed call with total silence, and elevated packet_loss in the CDR. Setting secure: true resolves it.

API equivalent

bashCopy
curl -X POST https://api.vobiz.ai/api/v1/Account/$AUTH_ID/trunks \
-H "X-Auth-ID: $AUTH_ID" \
-H "X-Auth-Token: $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "openai-realtime",
"trunk_direction": "inbound",
"trunk_status": "enabled",
"transport": "tls",
"secure": true,
"primary_uri_uuid": "<origination-uri-id>",
"concurrent_calls_limit": 10,
"cps_limit": 5
}'

bashCopy
curl -X PUT https://api.vobiz.ai/api/v1/Account/$AUTH_ID/trunks/$TRUNK_ID \
-H "X-Auth-ID: $AUTH_ID" \
-H "X-Auth-Token: $AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d '{"secure": true, "transport": "tls"}'

Step 3: Assign the phone number

Attach the DID that callers will dial to the trunk from Step 2.

bashCopy
# encode "+" as %2B
curl -X POST "https://api.vobiz.ai/api/v1/Account/$AUTH_ID/numbers/%2B919999999999/assign" \
  -H "X-Auth-ID: $AUTH_ID" \
  -H "X-Auth-Token: $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"trunk_group_id": ""}'

A successful assignment returns 204 No Content. See Assign Number to Trunk.

Step 4: Register the OpenAI webhook

platform.openai.com → Settings → Webhooks → Create

Copy the signing secret (whsec_…) shown once at creation. If it is lost, delete the webhook and recreate it.

Step 5: Configure the service

Environment variables

The repo ships a .env.example. Copy it with cp .env.example .env and fill in your values:

iniCopy
# ---- OpenAI ----
# API key from platform.openai.com
OPENAI_API_KEY=sk-proj-xxxxxxxx
# Project ID - must match the value used in the Origination URI
OPENAI_PROJECT_ID=proj_xxxxxxxx
# Signing secret shown when the webhook was created
OPENAI_WEBHOOK_SECRET=whsec_xxxxxxxx

# Bypass webhook signature verification. Development only - never enable in production.
SKIP_SIGNATURE_VALIDATION=false

# ---- Vobiz ----
# DID assigned to the inbound trunk, E.164
VOBIZ_PHONE_NUMBER=+919999999999

# ---- Service ----
SERVICE_PORT=8000

# ---- Agent ----
AI_MODEL=gpt-realtime-2.1
AI_VOICE=alloy
AI_INSTRUCTIONS=You are a helpful customer support agent.
AI_GREETING=Thank you for calling, how can I help?

# ---- Escalation (optional) ----
# Destination for AI-initiated transfer. tel: for E.164, or a SIP URI.
# Leave blank to disable the transfer tool.
REFER_TARGET=tel:+919888888888

Notes on the keys:

• Variable names are case-sensitive and must be uppercase.
• Names must not contain spaces. OPENAI WEBHOOK_SECRET is not a valid key and is silently ignored by python-dotenv, which surfaces later as a signature-verification failure.
• OPENAI_PROJECT_ID must be identical to the user part of the Origination URI. A mismatch routes the INVITE to a different project, and the webhook never fires.
• Your Vobiz AUTH_ID and AUTH_TOKEN are only needed for the provisioning calls in Steps 1-3. The service itself talks to OpenAI, not to the Vobiz API.

Dependencies

The repo's requirements.txt:

textCopy
flask>=3.0
openai>=1.99
websocket-client>=1.8
requests>=2.32
python-dotenv>=1.0
gunicorn>=22.0

Step 6: Handle the call

Three actions are required when realtime.call.incoming arrives. The excerpts below are from app.py in the sample repo.

Verify the webhook signature

pythonCopy
event = client.webhooks.unwrap(request.get_data(), dict(request.headers),
                               secret=WEBHOOK_SECRET)

Return 400 on failure. OpenAI signs each delivery with webhook-id, webhook-timestamp and webhook-signature headers.

Accept the call

Returning 200 to the webhook does not accept the call. Acceptance is a separate REST call:

pythonCopy
requests.post(
    f"https://api.openai.com/v1/realtime/calls/{call_id}/accept",
    headers={"Authorization": f"Bearer {API_KEY}",
             "Content-Type": "application/json"},
    json={
        "type": "realtime",
        "model": "gpt-realtime-2.1",
        "instructions": INSTRUCTIONS,
        "audio": {"output": {"voice": "alloy"}},
    },
)

Codecs are negotiated by OpenAI. Audio format fields are not set for SIP calls.

Open the control WebSocket

pythonCopy
ws.connect(f"wss://api.openai.com/v1/realtime?call_id={call_id}",
           header=[f"Authorization: Bearer {API_KEY}"],
           origin="https://api.openai.com")

This channel carries session events, transcripts and tool calls. Send response.create to make the agent speak first:

pythonCopy
ws.send(json.dumps({
    "type": "response.create",
    "response": {"instructions": f"Say to the user: {GREETING}"},
}))

Enable caller transcription (optional)

Caller-side transcription is off by default. Enable it after the socket opens:

pythonCopy
ws.send(json.dumps({
    "type": "session.update",
    "session": {
        "type": "realtime",
        "audio": {"input": {"transcription": {"model": "whisper-1"}}},
    },
}))

Caller speech then arrives as conversation.item.input_audio_transcription.completed, and agent speech as response.output_audio_transcript.done. Both carry a transcript field.

Transferring to a human agent

Escalation uses SIP REFER, issued by OpenAI and routed by the Vobiz trunk.

Give the agent a transfer tool

pythonCopy
ws.send(json.dumps({
    "type": "session.update",
    "session": {
        "type": "realtime",
        "tools": [{
            "type": "function",
            "name": "transfer_to_human",
            "description": "Transfer the caller to a human agent when they ask for a person.",
            "parameters": {
                "type": "object",
                "properties": {"reason": {"type": "string"}},
                "required": ["reason"],
            },
        }],
        "tool_choice": "auto",
    },
}))

Issue the REFER when the tool fires

Listen for response.function_call_arguments.done, then:

pythonCopy
requests.post(
    f"https://api.openai.com/v1/realtime/calls/{call_id}/refer",
    headers={"Authorization": f"Bearer {API_KEY}",
             "Content-Type": "application/json"},
    json={"target_uri": "tel:+919888888888"},
)

OpenAI sends a SIP REFER down the Vobiz trunk, which creates a new leg to the target and bridges the caller to it. The transferred leg appears in CDR as a separate outbound record carrying the trunk ID.

Transfer target requirements

• The target must be reachable and able to receive an INVITE.
• An endpoint that can place calls is not necessarily able to receive them. A registered SIP user requires an active contact binding in the registrar; without one, the INVITE cannot be delivered and the call clears instead of transferring.
• tel: targets take E.164 form, for example tel:+919888888888.

Other call controls

Running the service

Development

bashCopy
git clone https://github.com/vobiz-ai/Vobiz-Openai-Realtime
cd Vobiz-Openai-Realtime
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env    # fill in the values from Step 5
ngrok http 8000          # register the HTTPS URL as the webhook endpoint
python app.py

ngrok issues a new domain on each restart. The webhook URL in the OpenAI console must be updated whenever it changes.

Production

bashCopy
gunicorn -w 4 -b 0.0.0.0:8000 app:app

• Terminate HTTPS with a valid certificate - OpenAI does not deliver webhooks over plain HTTP.
• Keep SKIP_SIGNATURE_VALIDATION=false.
• Gunicorn workers do not share memory. Any per-call state must live in shared storage such as Redis if it is read outside the worker that accepted the call.

Verifying the integration

Webhook delivery

platform.openai.com → Settings → Webhooks → Send test event, selecting realtime.call.incoming. A correctly configured service returns 200.

Test events carry an empty data object and no live SIP session, so an accept attempt returns 404 call_id_not_found. The service should acknowledge this with 200 rather than an error - repeated 5xx responses count against the endpoint's delivery health.

Live call

Dial the assigned DID. The agent answers within a few seconds and speaks the configured greeting.

Behaviour reference

Useful CDR fields

Next steps

• Clone the sample service: vobiz-ai/Vobiz-Openai-Realtime.
• Explore the OpenAI Realtime API documentation.
• Review SIP trunk configuration and origination URIs.
• Inspect call quality and routing in the CDR reference.