Real-time Transport Protocol (RTP) jitter is the reason a phone call over the internet can sound perfectly clear one moment and glitchy the next — it's the one real-time medium on the internet that doesn't get a second chance.

A web page can take another 200ms to load and nobody notices. A video can buffer for a second. A file transfer can silently retransmit a dropped packet. Voice can't do any of that. The moment audio arrives late, out of order, or not at all, the human ear catches it instantly — as a robotic stutter, a gap, or an awkward pause mid-sentence.

For Voice AI this gets worse, not better. A delayed audio packet doesn't just distort what the listener hears, it delays speech recognition downstream, which makes the whole conversation feel unnatural and laggy. Understanding how production telephony systems fight this is one of the more underrated topics in real-time communications engineering.

Every RTP packet is supposed to arrive at a fixed, predictable interval. If you're running G.711 or Opus at 20ms packetization, the expectation looks like this:

Packet 1 : 20 ms
Packet 2 : 40 ms
Packet 3 : 60 ms
Packet 4 : 80 ms
...

Clean, evenly spaced, exactly 20ms apart. That's the theory.

In production, it almost never plays out that way. A real capture looks more like this:

Packet 1 : 20 ms
Packet 2 : 23 ms
Packet 3 : 41 ms
Packet 4 : 57 ms
Packet 5 : 91 ms
Packet 6 : 84 ms

Some packets show up early. Some show up late. Some arrive out of sequence. Some don't arrive at all. That variance in arrival timing is what we call jitter.

Jitter is almost never a codec problem. It's a network problem. The usual suspects are:

• Router queue congestion
• WiFi retransmissions
• LTE/5G scheduling delays
• MPLS rerouting
• Packets taking different paths across the network
• CPU scheduling delays on either end
• Virtual machine pauses
• General internet congestion

Think of it like highway traffic. Cars leave the toll booth at a steady 20-second interval, but traffic lights, merges, and congestion mean some arrive bunched together while others fall behind. RTP packets behave exactly the same way once they hit a congested network path.

Traditional telephony has one job: play the audio back. Voice AI has a much longer pipeline sitting behind that same audio stream. This is exactly where infrastructure choice matters.

RTP ──▶ Jitter Buffer ──▶ Decoder ──▶ Noise Cancellation / VAD
                                                           │
                                                           ▼
                       RTP ◀── Encoder ◀── TTS ◀── LLM ◀── ASR

Every single delayed packet delays everything downstream of it. A 40ms network delay doesn't stay a 40ms problem. It stacks:

• 40ms waiting in the jitter buffer
• 20ms of ASR delay
• 30ms of LLM scheduling
• 25ms of TTS generation

That's close to 100ms of added conversational latency from one network hiccup, and turn-taking is exactly where users feel it first.

Fixing this isn't about one clever algorithm or a bigger buffer, because no single fix covers every failure mode. That's why the eight approaches below are called layers rather than solutions: each operates at a different point in the pipeline — some on individual packets (reordering, concealment, correction), some on the buffer itself (static or adaptive), some on the network path (QoS, routing) — and production systems run all eight together rather than picking one.

The most straightforward fix is to intentionally delay playback so late packets get a chance to catch up.

Incoming Packets: 20, 40, 60, 80, 100
↓
Wait 60 ms
↓
Play

Instead of playing audio the moment it arrives, the receiver holds a few packets back and lets them accumulate first. That gives slower packets room to show up before playback needs them.

A quick example. Say packets arrive at 20ms, 40ms, 62ms, 79ms, and 100ms. Without any buffering, the packet due at 60ms never made it in time, so the listener hears a click:

20 ✓
40 ✓
60 Missing
80 Play

With a 60ms jitter buffer instead, by the time playback starts, the "late" packet has actually already arrived and is sitting in the buffer:

20 Store
40 Store
62 Store
79 Store
Playback begins: 20, 40, 60, 80

The click disappears entirely. The tradeoff: every conversation now carries some added latency, whether the network needed the buffer that day or not.

Static buffers have an obvious flaw: they're static, and networks are anything but. This is why virtually every modern VoIP stack has moved to adaptive buffering instead, where the system continuously estimates jitter and adjusts buffer size in real time.

When the network starts degrading, the buffer grows to compensate — 20ms → 40ms → 60ms → 80ms. When conditions stabilize again, it shrinks right back down to keep latency low — 80ms → 60ms → 40ms → 20ms.

How jitter actually gets measured: RFC 3550 defines this using an exponentially weighted moving average:

J = J + (|D(i-1,i)| - J) / 16

Where D is the variation in packet spacing and J is the running jitter estimate. The math is deliberately simple: it smooths out one-off spikes without overreacting, but still responds quickly if congestion is persistent rather than a fluke. Most RTP implementations you'll encounter in production run some variant of this exact formula.

Vobiz runs an adaptive buffer in production for exactly this reason, sized dynamically rather than fixed.

Packets don't always arrive in the order they were sent.

Received: 100, 120, 140, 160, 180
becomes:  100, 140, 120, 160, 180

The fix here is straightforward: the receiver holds packets briefly and re-sorts them using their RTP sequence numbers before handing them to the decoder. Skip this step and speech comes out audibly distorted, even if every packet technically arrived.

Vobiz applies packet reordering on every call, so out-of-sequence packets are corrected before they ever reach the decoder.

Some packets simply never show up, and you can't sit around waiting for them forever. So instead of waiting, modern codecs predict what the missing audio probably sounded like.

Take a word like this, with a chunk missing:

Hello Wo___d

The decoder reconstructs it using the speech characteristics that came just before the gap:

Hello World

For one or two dropped packets, most listeners never notice a thing. Opus has genuinely sophisticated PLC built in. G.711 implementations are cruder — they typically just repeat the previous audio frame or interpolate between samples, but it still beats dead silence or a harsh click.

Vobiz applies PLC across its calls, so brief packet loss gets concealed rather than heard as a gap.

Where PLC repairs damage after the fact, FEC tries to mitigate it, by sending redundant data alongside the original stream.

Packet 1, Packet 2, Packet 3, Parity Packet

If Packet 2 gets lost in transit, the receiver can reconstruct it directly from the parity data instead of concealing it:

Packet 1, Packet 3, Parity
↓
Recover Packet 2

Opus supports FEC natively. The tradeoff is bandwidth: you're always sending more data than the bare minimum, in exchange for the ability to recover from loss cleanly.

On Vobiz, FEC doesn't apply to the common trunk codecs (PCMU, PCMA, G.722) that carry most calls, since FEC is an Opus-specific feature. Concealment (PLC) still applies on those calls; correction (FEC) doesn't.

This one's subtle but important: playback timing should never be driven by when a packet arrives. It should be driven by the RTP timestamp embedded in the packet itself.

Because playback is scheduled against the timestamp rather than arrival time, a late packet still gets slotted into its correct position in the audio stream instead of playing out of sync. This is what keeps audio smooth even when arrival timing is genuinely erratic.

Vobiz schedules playback against RTP timestamps rather than arrival time, so a late packet still lands in its correct position instead of throwing off the rest of the stream.

No jitter buffer, however well-tuned, can fully compensate for a severely congested network. At some point you have to fix it at the transport layer instead. Carrier networks handle this by prioritizing RTP traffic ahead of everything else:

• DiffServ (DSCP EF)
• MPLS QoS
• VLAN priority
• Traffic shaping
• Queue scheduling

The net effect: voice packets get to jump the queue ahead of bulk data traffic instead of competing with it.

Vobiz marks outbound RTP with DSCP EF for priority handling, part of the same approach described above. Whether that marking is honored end-to-end still depends on every carrier and network hop along the path respecting it, something no platform can fully guarantee.

The last layer is at the platform level rather than the packet level. Large CPaaS providers actively monitor the quality of every network path in real time, and reroute around a path the moment it starts to degrade.

Carrier A → 40 ms jitter → Switch → Carrier B → 8 ms jitter

That routing decision isn't manual — it's driven continuously by packet loss, RTT, MOS, jitter, and historical quality.

Reducing jitter and reducing latency pull in opposite directions. You can't maximize both at once. The ITU-T G.114 recommendation caps acceptable one-way latency at 150ms for toll-quality voice, which is part of why production jitter buffers rarely grow past 100-120ms even on rough networks.

For voice AI specifically, most platforms deliberately lean toward the lower-latency end of that table, even if it means tolerating a slightly higher chance of packet loss, because a responsive conversation matters more than a perfectly clean one.

The same tradeoff plays out on Vobiz's own network, where the adaptive buffer stays lean by design rather than by accident.

Jitter cannot be fixed with one clever algorithm or a big buffer. It's eight layers working together:

• Stable network paths
• Adaptive jitter buffering
• Packet reordering
• Packet Loss Concealment (PLC)
• Forward Error Correction (FEC)
• RTP timestamp synchronization
• Quality of Service
• Intelligent media routing

For Voice AI, where every millisecond shows up directly in how natural a conversation feels, this stack is what separates a system that feels human from one that feels like a phone call from 2003.

Building Voice AI on infrastructure that handles jitter like this natively? Explore Vobiz's Programmable Voice API.