Skip to content

The Stack Beneath "Hello": How Your Voice Travels the Modern Internet

A voice becomes tens of thousands of packets per call, crosses unknown networks, and reassembles with so little delay our brains accept it as "live." This post follows a single "hello" down through history, protocols, the kernel, and out the other end into a machine that talks back.

Contents

  1. Introduction - the magic we stopped noticing
  2. From copper to code - a brief history
  3. The cast of characters - concepts and terminology
  4. Anatomy of a call - the happy path
  5. The protocol layer - rules, messages, invariants
  6. Going deeper - the network and OS layers
  7. The switchboard reimagined - PBX systems and bridges
  8. The next frontier - AI agents that pick up the phone
  9. Appendix - Installing Asterisk 22 from source on Ubuntu

1. Introduction - The Magic We Stopped Noticing

Pick up your phone. Dial someone halfway across the world. They say "hello", you say "hello" back. No glitch, no awkward gap, no robotic stutter. Feels like magic, but it is a magic we stopped noticing.

Here is what happens in those two seconds:

  • Your voice is sampled thousands of times per second, encoded by a codec, and cut into small chunks of 20 milliseconds each [^rfc3551].
  • Each chunk is wrapped in an RTP packet [^rfc3550] and sent over UDP [^rfc768], which makes no promise to deliver anything.
  • These packets travel across an Internet that does not know - and does not care - that they carry a human voice. Two packets sent back to back can take totally different routes.
  • On the other side, packets arrive out of order, some never arrive at all, and a jitter buffer quickly puts them back together so your ear never notices the mess.

A 5-minute call produces around 30,000 packets flying in both directions. And the whole thing has to finish inside a brutal budget: ~150 milliseconds, mouth to ear, one-way [^g114]. Past that, the call starts to feel laggy. Past 400 ms, it falls apart.

flowchart LR
    A["Microphone<br/>analog voice"] --> B["Codec<br/>20 ms chunks"]
    B --> C["RTP / UDP<br/>packets"]
    C --> D(("Internet<br/>best-effort"))
    D --> E["Jitter buffer<br/>reorder + smooth"]
    E --> F["Decoder"]
    F --> G["Speaker"]

The wild part is that this works at all. Modern voice rides best-effort packets across shared infrastructure originally built for email and cat pictures. No reserved path, no guarantee. The fact that it still feels live is the magic we stopped noticing.

In this post, we will follow a single "hello" from your mouth to the speaker on the other end. We will cover how telephony got here, what SIP and RTP actually do, why NAT is the network engineer's nightmare, what a PBX is doing in the middle, and what changes when one side of the call is an AI agent instead of a person.

[^rfc3550]: RFC 3550 - RTP: A Transport Protocol for Real-Time Applications, H. Schulzrinne et al., 2003. [^rfc3551]: RFC 3551 - RTP Profile for Audio and Video Conferences with Minimal Control, §4.5 on packetization intervals. [^rfc768]: RFC 768 - User Datagram Protocol, J. Postel, 1980. [^g114]: ITU-T Recommendation G.114 (05/2003) - One-way transmission time. Defines the ≤150 ms / 150-400 ms / >400 ms quality bands for conversational voice.

2. From Copper to Code - A Brief History

Before VoIP existed, there was the PSTN - the Public Switched Telephone Network. For most of the 20th century, it ran on copper wires going from your house to a local telephone exchange, and from there to other exchanges all over the country and the world [^pstn].

When you spoke into the handset, the microphone turned your voice into a small changing electrical current. That current traveled down the copper wire as an analog signal - basically a moving picture of your voice's sound wave, drawn in electricity. On the other end, the speaker turned the same current back into sound. No digitization, no packets, no encoding - the wire carried the voice directly.

To connect two people, the network had to physically build a path - through relays and switches - between their two phones, and hold that path open for the entire call. This is called circuit switching, and it has two important properties:

  • The path is dedicated. While the call is up, those wires and switch contacts belong to you. Nobody else can use that exact path.
  • The quality is predictable. The line is open or it is not. There is no "the network got busy and your audio glitched."
flowchart TB
    subgraph CS["Circuit-switched (PSTN)"]
        direction LR
        A1["Phone A"] === S1["Exchange 1"] === S2["Exchange 2"] === A2["Phone B"]
    end
    subgraph PS["Packet-switched (VoIP)"]
        direction LR
        B1["Phone A"] -- "pkt 1" --> R1["Router"]
        B1 -- "pkt 2" --> R2["Router"]
        B1 -- "pkt 3" --> R3["Router"]
        R1 --> B2["Phone B"]
        R2 --> B2
        R3 --> B2
    end

The cost of circuit switching was a huge amount of wiring and big switching equipment. The benefit was steady, predictable quality.

For a long time, even the long-distance backbone was analog. Starting in the 1960s, the trunks between exchanges began to get digitized. The analog voice was sampled 8,000 times per second and turned into an 8-bit number per sample. This is the G.711 codec, also known as PCM [^g711]. The resulting 64 kbit/s digital stream was easier to multiplex, regenerate, and route over long distances. But the local loop to your house - the last copper mile - stayed analog for decades.

The Packet-Switched Leap

Voice and the Internet grew up separately. The Internet was built for data: emails, files, web pages. It was packet-switched from the start, meaning data is broken into small packets and each packet finds its own way through a shared network. Two packets in the same message can take different paths. The network does not reserve anything for anyone.

For data, this is fine. A web page does not care if one packet arrives 10 ms after another - the browser waits and reassembles. For voice, this is a real problem. Voice is real-time. A late packet is a useless packet. The whole reason circuit switching existed was to avoid this problem.

And yet, in the mid-1990s, people started shipping voice over the Internet anyway. The first widely known consumer product was VocalTec Internet Phone in 1995 [^vocaltec]. It was choppy and weird and worked over dial-up, but it proved the concept: voice could ride on packets, and you could skip the long-distance phone bill while doing it.

Two Inflection Points: SIP and Asterisk

Two things turned VoIP from a curiosity into infrastructure.

SIP (Session Initiation Protocol) was first published as a standard in 1999 as RFC 2543, and replaced by RFC 3261 in 2002 [^rfc3261]. Before SIP, every VoIP product spoke its own dialect, so two products from different vendors could not talk to each other. SIP gave the industry a common, open language for "who is calling whom, are you ringing, did you answer, did you hang up." It was designed to look like HTTP on purpose, so engineers who already knew the web could read SIP messages and understand them.

Asterisk appeared the same year, 1999, written by Mark Spencer [^asterisk]. Before Asterisk, a PBX - the switchboard that routes calls inside a company - was a proprietary box from a vendor like Cisco, Avaya, or Nortel, and it cost tens of thousands of dollars. Asterisk turned the PBX into a piece of open-source software you could run on a normal Linux PC. Suddenly, anyone could build phone systems.

Together, these two pieces created the world we live in now. Voice runs on packet networks, signaling is standardized and open, and the PBX in the middle is software. Packet networks were never designed for real-time voice. Yet here we are.

[^pstn]: For the analog local loop, circuit switching, and the broader PSTN history, see the ITU-T Q.700-series (SS7 / PSTN signaling overview) and the Bell System Technical Journal archives. [^g711]: ITU-T Recommendation G.711 (1972) - Pulse Code Modulation (PCM) of voice frequencies. The classic 64 kbit/s narrowband codec used across PSTN trunks. [^vocaltec]: VocalTec, Internet Phone, released February 1995. Widely cited as the first consumer VoIP product. [^rfc3261]: RFC 3261 - SIP: Session Initiation Protocol, J. Rosenberg et al., June 2002. Replaces RFC 2543 (1999). [^asterisk]: Asterisk was created by Mark Spencer in 1999 and is now maintained by Sangoma. See https://www.asterisk.org/ for the project's official history.

3. The Cast of Characters - Concepts and Terminology

Before we go deeper, we need to fix the vocabulary. People who talk about VoIP throw four or five acronyms into the same sentence, and they often mean very different things. Let me untangle them one by one.

Simple SIP call flow between two phones

VoIP - the umbrella

VoIP stands for Voice over IP. It is not a protocol. It is not a product. It is just the idea of carrying voice as packets over an IP network, instead of over circuit-switched copper. Every other term in this chapter lives inside VoIP.

SIP - the signaling

SIP stands for Session Initiation Protocol. It is a real, open protocol with its own RFC - RFC 3261, published by the IETF in 2002 [^rfc3261]. Just like HTTP defines how browsers and web servers talk, SIP defines how phones, PBXes, and other VoIP devices talk about a call.

What does SIP actually carry? Not the audio. SIP handles all the "around the call" stuff:

  • "Hey, I want to call you." (INVITE)
  • "Your phone is ringing." (180 Ringing)
  • "I picked up." (200 OK)
  • "I am hanging up." (BYE)

Think of SIP as the doorbell, the knock, and the goodbye wave. It does not carry the conversation itself.

RTP - the media

RTP stands for Real-time Transport Protocol [^rfc3550]. RTP carries the actual audio. Once SIP has set up the call - "I am ringing, you picked up, here are the codecs we agreed on" - RTP starts flowing between the two endpoints. Each RTP packet is one of those 20 ms chunks of voice we saw in chapter 1.

RTP runs on top of UDP. That raises a fair question: why not just send the audio over UDP directly? Because raw UDP gives the receiver nothing but a payload. For voice you need to know what order each chunk goes in (UDP can deliver packets out of order), when each chunk should be played (the jitter buffer needs a timestamp), which codec decoded it, and who sent it (a conference call has multiple senders). RTP adds a small 12-byte header that carries exactly those four things: a sequence number, a timestamp tied to the codec's sample clock, a payload type that names the codec, and an SSRC that identifies the source [^rtp-header]. Without that header, the receiver could not reassemble or play the call.

RTP is the conversation. SIP is everything around it.

Signaling vs Media - the most important split

This is the single biggest idea in the whole post: signaling (SIP) and media (RTP) are two separate channels. They do not travel together. They can use different ports, take different routes through the network, and even pass through different middleboxes.

A call setup looks roughly like this:

  1. Your phone talks to a SIP server (or another phone) using SIP.
  2. SIP messages negotiate where the audio will go. The actual negotiation - what codecs, which IP, which port - rides inside SIP as a small text blob called SDP (Session Description Protocol). We will read a real SDP body in chapter 4.
  3. Once both sides agree, RTP starts flowing directly between the two endpoints, often not even passing through the SIP server.

We will see in chapter 6 why this split matters so much - especially when NAT routers get involved.

PBX - the switchboard

PBX stands for Private Branch Exchange. It is a piece of equipment - or in the modern world, software like Asterisk - that sits in the middle and routes calls within an organization. If you call a company and hear "press 1 for sales, press 2 for support", that is a PBX. If your call gets transferred to another extension, that is also a PBX.

In VoIP terms, a PBX talks SIP to your phone, talks SIP to the outside world, and handles the routing in the middle. It can also sit in the media path - we will get into that in chapter 7.

Codecs - turning voice into bytes

A codec is the algorithm that turns analog voice into bytes and back. Every codec balances three things: audio quality, bandwidth, and CPU cost. The three you will keep running into:

Codec Bandwidth Sample rate Quality CPU Where you find it
G.711 64 kbit/s 8 kHz Toll quality (PSTN ref) Near zero PSTN bridging, anywhere bandwidth cheap
G.729 8 kbit/s 8 kHz Lower than G.711 Moderate (CELP) Old enterprise PBXes, slow WAN links
Opus 6 - 510 kbit/s 8 - 48 kHz Best in class, adaptive Higher, still trivial WebRTC, modern softphones, Internet

G.711 [^rfc3551] is the floor every VoIP system supports. No compression - each 8-bit sample maps directly to an amplitude. Two flavors: µ-law (PCMU, used in the US and Japan) and A-law (PCMA, used in Europe and most of the world). Quality matches PSTN exactly because G.711 is the PSTN encoding.

G.729 is the classic compressed codec, 8 kbit/s using CELP (code-excited linear prediction). Built for the era when WAN links cost real money.

Opus [^opus] is the modern default. It is adaptive - it changes bitrate and audio bandwidth on the fly based on network conditions and content type. Speech and music both. Default audio codec for WebRTC.

Codec choice is negotiated in SDP. A call between an old desk phone (G.711 only) and a WebRTC browser (Opus preferred) will land on G.711, because that is the only codec both sides understand. When two sides cannot agree, a B2BUA in the middle (chapter 7) can transcode between codecs - at the cost of CPU and added latency.

WebRTC - voice in the browser

A quick word on WebRTC, because the term will keep coming up. WebRTC is not a competitor to SIP. It is a set of browser APIs and protocols that let a web page do real-time audio and video without any plugin [^rfc8825]. The W3C defines the JavaScript API; the IETF defines the wire protocols.

Here is what is interesting: WebRTC uses the same RTP and the same codecs as SIP-based VoIP. The audio packets a browser sends are RTP packets [^rfc8825]. What WebRTC does not mandate is the signaling channel - the browser app is free to use SIP-over-WebSockets, or its own JSON-over-HTTP, or anything else, to exchange SDP bodies. The split between "set up the call" and "send the media" is the same; only the signaling layer differs.

In practice, a modern AI voice agent sitting in the browser is almost certainly using WebRTC + Opus on the client side, even if there is a SIP/Asterisk PBX bridging the call back to the PSTN further down the line.

Supporting cast

A few more terms that show up everywhere:

  • User Agent - any SIP-speaking endpoint. Your softphone, your desk phone, an Asterisk server - all of them are user agents [^rfc3261]. There are two roles: UAC (User Agent Client - sends the request) and UAS (User Agent Server - responds). The same device switches between these two roles depending on whether it is calling or being called.
  • Softphone - a phone implemented in software. Examples: Zoiper, Linphone, MicroSIP.
  • Endpoint - any device or app that can make or receive calls.
  • Trunk - a SIP connection between two providers, or between your PBX and your phone carrier. A trunk is a pipe that carries many calls at once.

That is the cast. Now we can start watching them act.

[^rtp-header]: RFC 3550 §5.1 - the 12-byte RTP fixed header: sequence number (16 bits), timestamp (32 bits), payload type (7 bits), and SSRC (32 bits), among other fields. [^rfc3551]: RFC 3551 - RTP Profile for Audio and Video Conferences with Minimal Control. §4.5 defines the standard 20 ms packetization for narrowband audio; payload types 0 (PCMU) and 8 (PCMA) define G.711. [^opus]: RFC 6716 - Definition of the Opus Audio Codec. Mandated for WebRTC by RFC 7874. [^rfc8825]: RFC 8825 - Overview: Real-Time Protocols for Browser-Based Applications. WebRTC media uses RTP with the SAVPF profile (RFC 5124).

4. Anatomy of a Call - The Happy Path

Now that the cast is on stage, let's watch a single call from start to finish. Alice calls Bob. Both run softphones registered to the same SIP server. We will walk through what each party sends and receives, in order, with no error paths and no surprises - the "happy path."

SIP session setup between Alice and Bob

Registration: how the network finds you

Before anyone can call Alice, the network has to know where Alice is right now. Her softphone, when it starts up, sends a SIP REGISTER request to the SIP server. The message says, in effect, "I am alice@example.com and I am currently reachable at IP 192.0.2.10, port 5060. Please remember this for the next hour."

The SIP server stores this mapping in its registrar [^rfc3261]. Bob's phone does the same when it starts up. Now both phones are findable by their SIP address (alice@example.com, bob@example.com), regardless of where in the world they actually sit.

This is the SIP equivalent of DNS for phones.

The INVITE and the SDP offer

Alice dials Bob. Her phone sends a SIP INVITE to the SIP server. The INVITE looks somewhat like an HTTP request - a method, headers, and a body. The body is an SDP offer. SDP stands for Session Description Protocol [^rfc4566], and it is the text format SIP uses to describe what kind of media session the two parties will set up.

A real SDP offer body looks like this:

v=0
o=alice 2890844526 2890842807 IN IP4 192.0.2.10
s=Call
c=IN IP4 192.0.2.10
t=0 0
m=audio 49170 RTP/AVP 0 8
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000

Reading it line by line:

  • c=IN IP4 192.0.2.10 - "send media to me at this IP."
  • m=audio 49170 RTP/AVP 0 8 - "on UDP port 49170, RTP/AVP profile, with payload types 0 and 8."
  • a=rtpmap:0 PCMU/8000 and a=rtpmap:8 PCMA/8000 - "payload type 0 is G.711 µ-law at 8 kHz; payload type 8 is G.711 A-law."

Alice is telling Bob: "I can speak two codecs, both G.711. Send your audio to me here."

The SIP server forwards the INVITE to Bob's phone using the IP it remembers from REGISTER.

Ringing, answer, ACK

Bob's phone receives the INVITE and replies with a sequence of SIP responses:

  1. 100 Trying - "I got your INVITE, I am working on it." (Provisional. Stops the network from retransmitting.)
  2. 180 Ringing - "Bob's phone is ringing right now." (Alice hears a ringback tone at this point.)
  3. 200 OK - "Bob picked up." The 200 OK body contains Bob's SDP answer: which of the offered codecs he picked, and where he wants Alice to send her audio.
  4. Alice's phone replies with ACK - "Got it, we are good." This three-way handshake confirms both sides agree the call is up [^rfc3261].

The combination of Alice's offer and Bob's answer is the SDP offer/answer model [^rfc3264]. After this exchange, both endpoints know exactly where to send RTP and what to encode.

Media flows

The moment the 200 OK lands, RTP starts flowing, directly between Alice's phone and Bob's phone - usually not through the SIP server at all. This is the signaling-vs-media split from chapter 3 made concrete: SIP set up the meeting, then stepped out of the room.

Each RTP packet is a 20 ms chunk of voice, with its sequence number, timestamp, and payload type. Fifty packets per second per direction. The jitter buffer on each side smooths the arrival times. Alice and Bob hear each other live.

Mid-call changes: re-INVITE

Real calls are not static. People press hold. They transfer. They roam from Wi-Fi to mobile and the IP address changes. SIP handles all of this with a re-INVITE - an INVITE sent inside an existing dialog, carrying a new SDP offer. The other side answers with a new SDP, and the media path adjusts. The call keeps going.

This is the second reason the SIP server stays in the picture even after media starts: someone has to carry the re-INVITE.

BYE and teardown

When Alice hangs up, her phone sends a BYE over SIP. Bob's phone replies 200 OK and tears down its end. RTP stops on both sides.

If the SIP server (or any signaling proxy) vanished mid-call, the BYE might never arrive. The other phone would keep sending RTP into the void until a media timer or RTCP feedback declared the call dead. This is the first reason the SIP server stays in the signaling path: somebody has to deliver the hangup.

The sequence, all in one place

sequenceDiagram
    participant A as Alice (UAC)
    participant S as SIP Server
    participant B as Bob (UAS)
    A->>S: REGISTER
    S-->>A: 200 OK
    B->>S: REGISTER
    S-->>B: 200 OK
    A->>S: INVITE (SDP offer)
    S->>B: INVITE (SDP offer)
    B-->>S: 100 Trying
    B-->>S: 180 Ringing
    S-->>A: 180 Ringing
    B-->>S: 200 OK (SDP answer)
    S-->>A: 200 OK (SDP answer)
    A->>B: ACK
    Note over A,B: RTP media flows directly<br/>between Alice and Bob
    A->>B: BYE
    B-->>A: 200 OK

That is the whole happy path. Every detail in the rest of the post is either an expansion of one of these steps, or a problem that comes from the network not cooperating.

[^rfc4566]: RFC 4566 - SDP: Session Description Protocol, M. Handley et al., July 2006. [^rfc3264]: RFC 3264 - An Offer/Answer Model with the Session Description Protocol (SDP).

5. The Protocol Layer - Rules, Messages, and Invariants

If chapter 4 was the story of a call, this chapter is the grammar - the rules that make the story possible. Newcomers can skim this one and come back later; engineers who read SIP traces every day live here.

SIP is text, and looks like HTTP on purpose

SIP messages are plain text, just like HTTP requests. You can read them with your eyes. Here is what an actual INVITE looks like on the wire:

INVITE sip:bob@example.com SIP/2.0
Via: SIP/2.0/UDP 192.0.2.10:5060;branch=z9hG4bK776asdhds
Max-Forwards: 70
From: Alice <sip:alice@example.com>;tag=1928301774
To: Bob <sip:bob@example.com>
Call-ID: a84b4c76e66710@192.0.2.10
CSeq: 314159 INVITE
Contact: <sip:alice@192.0.2.10>
Content-Type: application/sdp
Content-Length: 142

v=0
o=alice 2890844526 2890842807 IN IP4 192.0.2.10
s=Call
c=IN IP4 192.0.2.10
t=0 0
m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000

Compare it to an HTTP request:

POST /some/resource HTTP/1.1
Host: example.com
Content-Type: application/json
Content-Length: 42

{ ... }

Almost the same shape. Request line on top, then headers, then a blank line, then the body. The IETF chose this shape on purpose, so engineers already fluent in HTTP could read SIP messages without learning a whole new format [^rfc3261-design].

A few headers do the real work in SIP:

  • Call-ID - a globally unique identifier for this entire call. Stays the same across INVITE, 200 OK, ACK, and BYE.
  • CSeq - command sequence number; increments per new request, used to match responses to their requests.
  • From / To - logical caller and callee. Each has a tag parameter (tag=1928301774) that helps identify the dialog (see below).
  • Via - records the route the request took. Responses follow the Via chain backwards to find Alice.
  • Contact - direct address to use for subsequent requests inside this call.

Transactions vs dialogs

These two terms confuse everyone at first, but they are the load-bearing concepts of SIP. They define what it means for a call to "exist."

A transaction is one request plus all its responses [^rfc3261-tx]. The INVITE and the 100 Trying, 180 Ringing, 200 OK it triggers - that is one transaction. BYE and its 200 OK is a different transaction. Transactions are short-lived. They live long enough to deliver a request and collect its responses, then they are done. If a response is lost in flight, the transaction layer retransmits the request, because SIP runs over unreliable UDP and cannot trust the network.

A dialog is the longer-lived relationship between two endpoints during a call [^rfc3261-dialog]. A dialog spans many transactions: the initial INVITE/200/ACK that creates it, any re-INVITEs in the middle, and the BYE that destroys it. A dialog is uniquely identified by three things: the Call-ID, the local tag, and the remote tag. As long as those three values stay the same on both sides, both endpoints agree they are in the same call.

The mental model:

  • Dialog = the conversation as a whole, from "hello" to "goodbye."
  • Transaction = a single exchange inside that conversation - "send INVITE, collect responses", "send BYE, collect response."

This is the SIP equivalent of a TCP connection (the long-lived thing) and an HTTP request inside it (the short-lived thing). Different scope, different lifetime.

SDP rides inside SIP, but is not SIP

We met SDP in chapter 4 as the text blob that describes the media session. Strictly speaking, SDP is its own protocol [^rfc4566]. SIP just carries it, the way HTTP carries JSON in a body. This is a clean separation of concerns:

  • SIP knows how to deliver messages between endpoints. It does not care what is inside the body.
  • SDP knows how to describe a media session. It does not care how the description gets transported.

You can find SDP carried inside other protocols too - RTSP for streaming, for example. And SIP can carry other body types: T.140 for real-time text, ISUP for PSTN interop. Keeping the two layers separate means each one can evolve on its own.

RTCP - the quality feedback channel

RTP has a twin that often gets forgotten: RTCP, the RTP Control Protocol [^rfc3550-rtcp]. While RTP carries the audio, RTCP runs alongside it on the next UDP port up (by convention, RTP on the even port, RTCP on the next odd one) and carries statistics in both directions:

  • How many packets did I receive?
  • How many did I lose?
  • What was the jitter?
  • How long since I last heard from you?

Endpoints exchange RTCP reports every few seconds. The data is what monitoring tools - Wireshark, Asterisk's own call detail records, network probes - use to score call quality. MOS scores, jitter graphs, and packet loss percentages all come from RTCP.

RTCP packets are tiny. By default, RTCP is allowed at most 5% of the RTP bandwidth, so it never crowds out the audio [^rfc3550-rtcp].

The core invariant: signaling vs media reliability

Everything we have built up to comes down to a single asymmetry.

Signaling can be lossy and retried. A lost INVITE is just a delayed call setup; the transaction layer retransmits it. A lost BYE is annoying but recoverable - a re-send, or a media timeout, cleans the call up. SIP can run over UDP, TCP, or TLS, and the protocol assumes packets may be lost.

Media cannot wait. A retransmitted audio packet is a useless audio packet, because by the time it arrives, the moment has passed [^rfc3550]. The right answer for lost media is to drop it, paper over the gap with packet loss concealment, and keep going. This is why media uses UDP and never TCP - TCP's retransmissions would cause the very latency they are trying to recover from.

That single asymmetry dictates almost every design choice in the next chapter:

  • Media over UDP, signaling free to run over TCP or TLS.
  • Jitter buffer on media, retransmission on signaling.
  • DSCP marking favors media over signaling.
  • NAT traversal is harder for media (long-lived UDP flows) than for signaling (request/response).

Once you internalize this asymmetry, the rest of the stack stops feeling arbitrary.

[^rfc3261-design]: RFC 3261 §7 describes the SIP message format, which mirrors HTTP/1.1 (RFC 2616) message syntax on purpose. [^rfc3261-tx]: RFC 3261 §17 - Transactions. Defines the client and server transaction state machines. [^rfc3261-dialog]: RFC 3261 §12 - Dialogs. Defines dialog identification via Call-ID + local tag + remote tag. [^rfc3550-rtcp]: RFC 3550 §6 - RTP Control Protocol (RTCP). Defines Sender Report / Receiver Report packets and the default 5% bandwidth cap.

6. Going Deeper - The Network and OS Layers

Where the internet's indifference to your phone call becomes a real engineering problem.

  • UDP vs. TCP - media uses UDP; late packets are worthless, so don't retransmit.
  • Jitter and jitter buffers - smoothing arrival variance.
  • Latency budget - ~150 ms one-way before conversation degrades.
  • Packet loss concealment - what codecs do when bytes vanish.
  • NAT traversal - the famous wound. SIP's in-message addresses break behind NAT; STUN/TURN/ICE is the cure.
  • QoS / DSCP marking - telling the network this packet is special.
  • OS layer - audio capture from the mic, kernel network stack, scheduling. Jitter is partly an OS problem, not just a network one.

Diagram 3 - signaling vs. media path split: SIP flowing through proxies/PBX, RTP taking a direct (or relayed) path, NAT boxes in between.

7. The Switchboard Reimagined - PBX Systems and Bridges

Everything so far assumed two endpoints talking directly. Real systems almost never work that way.

A PBX is not a passive relay - it's a B2BUA (back-to-back user agent): it terminates one call leg and originates another, sitting in the middle of both signaling and (often) media. Concretely, Asterisk and FreeSWITCH do:

  • Call routing and dialplans.
  • IVR (interactive voice response).
  • Conferencing as media mixing.
  • Transcoding between codecs.
  • Bridging VoIP ↔ legacy PSTN.

Putting a box in the media path costs latency and capacity, but buys control, recording, and interop. This is where the rest of the post pays off in practice.

8. The Next Frontier - AI Agents That Pick Up the Phone

The pipeline:

RTP audio → STT → LLM → TTS → RTP back

…bolted onto the exact SIP/PBX infrastructure described above. The brutal new constraint: the ~150 ms latency budget from chapter 6 now has to also contain model inference. Real-time voice APIs collapse the pipeline into a single streaming model.

What changes philosophically when one or both parties is a machine? The miracle from the intro - voice across packet networks - now has a machine on the line.

9. Appendix - Installing Asterisk 22 from Source on Ubuntu

This appendix is the lab companion to the post. Target: a fresh Ubuntu 22.04 VPS. Goal: working Asterisk 22 with PJSIP and ARI, ready to bridge to an external AI service.

9.1 Prerequisites

Become root and update the base system:

whoami

apt update
apt upgrade -y

apt install -y wget curl tar nano build-essential

9.2 Fetch and Extract Source

Move the source archive to a normal source-build location:

mkdir -p /usr/local/src
mv /root/asterisk-22-current.tar.gz /usr/local/src/
cd /usr/local/src

tar -xzf asterisk-22-current.tar.gz
ls -la

You should now see a directory like asterisk-22.x.x. Enter it:

cd asterisk-22*/
pwd
ls -la configure contrib/scripts/install_prereq

Expected:

configure
contrib/scripts/install_prereq

9.3 Install Build Dependencies

Asterisk ships its own prereq installer:

contrib/scripts/install_prereq install

This is the part the official docs refer to. (docs.asterisk.org)

Optional, only if you need MP3 music-on-hold support:

contrib/scripts/get_mp3_source.sh

9.4 Configure and Select Modules

./configure

For Asterisk 15+, bundled pjproject is enabled by default, so plain ./configure is normally enough for PJSIP usage. (docs.asterisk.org)

If make menuselect complains, install its deps:

apt install -y libncurses-dev libnewt-dev

Open module selection:

make menuselect

For an OnlinePBX ↔ Asterisk ↔ AI-bridge ↔ TTS provider topology, make sure these are enabled if visible:

Resource Modules:
  res_pjsip
  res_pjsip_session
  res_pjsip_transport_websocket
  res_ari
  res_http_websocket

Channel Drivers:
  chan_pjsip

Applications:
  app_dial
  app_playback
  app_record
  app_stasis

Codec Translators:
  codec_ulaw
  codec_alaw
  codec_opus     # if available/needed

Save and exit.

9.5 Build and Install

make -j"$(nproc)"
make install
make samples        # sample configs; first-boot only, not production-safe
make config         # systemd service files
ldconfig

9.6 Dedicated User and Permissions

Create the asterisk user if missing:

id asterisk || adduser --system --group --home /var/lib/asterisk --no-create-home asterisk

Set ownership:

chown -R asterisk:asterisk /etc/asterisk
chown -R asterisk:asterisk /var/lib/asterisk
chown -R asterisk:asterisk /var/log/asterisk
chown -R asterisk:asterisk /var/spool/asterisk
chown -R asterisk:asterisk /usr/lib/asterisk 2>/dev/null || true
chown -R asterisk:asterisk /usr/lib64/asterisk 2>/dev/null || true

Tighten config perms:

find /etc/asterisk -type d -exec chmod 750 {} \;
find /etc/asterisk -type f -exec chmod 640 {} \;

Tell Asterisk to run as the asterisk user - edit /etc/asterisk/asterisk.conf:

[options]
runuser = asterisk
rungroup = asterisk

9.7 Enable and Start

systemctl daemon-reload
systemctl enable asterisk
systemctl start asterisk

systemctl status asterisk --no-pager

Verify it is not running as root:

ps -eo user,group,cmd | grep '[a]sterisk'

Expected: user/group asterisk.

9.8 Smoke Test via CLI

asterisk -rvvv

Inside the CLI:

core show version
module show like pjsip
http show status
ari show status
exit

If systemctl start asterisk fails:

journalctl -u asterisk -xe --no-pager
tail -n 100 /var/log/asterisk/messages

9.9 Minimum Firewall (SIP + RTP)

apt install -y ufw

ufw allow OpenSSH
ufw allow 5060/udp
ufw allow 10000:20000/udp
ufw enable
ufw status verbose

9.10 Lock Down ARI / HTTP

Do not expose AMI/ARI publicly. Bind to localhost first.

/etc/asterisk/http.conf:

[general]
enabled=yes
bindaddr=127.0.0.1
bindport=8088

/etc/asterisk/ari.conf:

[general]
enabled = yes
pretty = yes

[ai_bridge]
type = user
read_only = no
password = CHANGE_THIS_TO_A_LONG_RANDOM_PASSWORD

Restart and check:

systemctl restart asterisk
asterisk -rx "ari show users"
asterisk -rx "http show status"

9.11 Full Sequence (Copy-Paste)

From a clean Ubuntu VPS with the tarball already in /root:

apt update
apt upgrade -y
apt install -y wget curl tar nano build-essential

mkdir -p /usr/local/src
mv /root/asterisk-22-current.tar.gz /usr/local/src/
cd /usr/local/src
tar -xzf asterisk-22-current.tar.gz
cd asterisk-22*/

contrib/scripts/install_prereq install
./configure
make menuselect
make -j"$(nproc)"
make install
make samples
make config
ldconfig

id asterisk || adduser --system --group --home /var/lib/asterisk --no-create-home asterisk

chown -R asterisk:asterisk /etc/asterisk /var/lib/asterisk /var/log/asterisk /var/spool/asterisk
find /etc/asterisk -type d -exec chmod 750 {} \;
find /etc/asterisk -type f -exec chmod 640 {} \;

printf '\n[options]\nrunuser = asterisk\nrungroup = asterisk\n' >> /etc/asterisk/asterisk.conf

systemctl daemon-reload
systemctl enable asterisk
systemctl start asterisk

systemctl status asterisk --no-pager
ps -eo user,group,cmd | grep '[a]sterisk'
asterisk -rvvv

Do not proceed to trunk credentials until core show version and module show like pjsip work cleanly inside the Asterisk CLI.