Low Latency Live Streaming: Why WebRTC
Every live stream runs a little behind the moment it shows. The camera sees the host wave, and viewers see the wave some time later: that gap is the latency, or delay, of the stream. For a stream that nobody answers, a few seconds do not matter. For a stream whose audience talks back, a Q&A, a class, a launch with questions, an auction, the delay decides whether the chat is a conversation or a string of answers to questions the host has already moved past.
This post goes through where the delay comes from, why the usual way of delivering video runs many seconds behind, and how WebRTC, the technology of video calls, brings it down. It is also how Live Stream delivers its streams.
Where the delay comes from
A live picture passes through five stages, and each adds to the delay:
- Capture and encoding. The camera takes the frame, and the encoder, a browser or OBS, compresses it.
- The way in. The stream travels to a server.
- The server. It may change the stream into other sizes or formats before passing it on.
- The way out. The stream travels to each viewer.
- The player. It holds some of the stream before playing it, a buffer, so that a late packet does not stop the picture.
The network part is usually the small part: a fraction of a second, even across an ocean. The big differences come from what the server does with the stream, and how much the player holds.
HLS and DASH: built for scale
Most video on the web is delivered with HLS or DASH. The server cuts the stream into short files, segments of a few seconds each, and lists them in a playlist. The player downloads one segment after another, like any other file, and holds a few of them before it starts, so the picture never runs dry.
That is why it scales so well: segments are plain files, which a CDN caches near every viewer, for any number of them. It is also why it runs behind. A segment exists only once its seconds have been recorded, and the player waits for several. An HLS stream often plays ten seconds or more behind the host. Low-Latency HLS and Low-Latency DASH cut segments into smaller parts and bring it down to a few seconds.
RTMP: the way in, not the way out
RTMP is how encoders have sent streams to servers for years. It once played in browsers too, through Flash, and today it carries the stream only to the server, which repackages it for viewers as HLS. So an RTMP workflow has the delay of HLS.
WebRTC: built for conversation
WebRTC is what browsers use for video calls, where a delay of seconds would make talking impossible. It sends the picture as a stream of small packets, which the receiving side plays as they arrive, holding only enough to smooth out the network. There are no segments to wait for.
For a stream with an audience, a media server sits in the middle: the host sends one stream to it, and it passes the packets on to each viewer. Two techniques make that work for many viewers with different screens and connections:
- Layers. The host sends the picture in several sizes at once, such as 720p, 540p and 360p, and the media server gives each viewer the size their player and connection need. Nothing is re-encoded on the server, so nothing waits for it.
- Repair. When a packet is lost, the viewer asks for it again, and the media server keeps the last moments of the stream to resend it.
The cost is on the side of whoever runs the media servers. A CDN cannot cache a WebRTC stream: every viewer has a live connection, and the servers must be close enough to the audience and large enough for it.
HLS and WebRTC side by side
| HLS and DASH | Low-Latency HLS and DASH | WebRTC | |
|---|---|---|---|
| Delay behind the host | Often ten seconds or more | A few seconds | About a second |
| Delivered by | Any CDN, as files | A CDN that supports it | Media servers, a connection for each viewer |
| Rewinding and catching up | Yes | Yes | No, always at the live moment |
| A chat in step with the picture | No | Almost | Yes |
| Plays in | Every browser, with a player script in some | Every browser, with a player script in some | Every current browser |
How Live Stream streams over WebRTC
Live Stream delivers every stream over WebRTC, through the media servers of WordPlus Cloud:
- The host sends in layers. A host going live from the browser sends the picture in several sizes, and OBS sends up to three with its Total Layers setting (How to stream from OBS to WordPress over WHIP). Each viewer gets the size their player and connection need, so a phone gets less than a big screen.
- The camera is checked as it goes live. The browser of the host tries H.264 first, which plays on the most devices, and moves to VP9, then VP8, if the computer cannot keep up with it.
- Each viewer holds about a second while their connection plays smoothly. If the picture freezes while packets of it are lost, the player holds two seconds, then three, and comes back down after minutes without trouble. The sound follows the picture, so lips and voice stay together. The Statistics of the player show the buffer as it changes.
- No app. Viewers watch on the page of your site, in the browser.
The whole delay is that buffer plus the way through the network, so on a smooth connection viewers stay within a couple of seconds of the host, and the chat beside the picture is about what is on screen.
When HLS is the better choice
WebRTC is not always the answer:
- Rewinding. HLS players can go back to an earlier moment of a live stream. A WebRTC stream is always at the live moment.
- A passive audience of millions. If nobody needs to talk back, files on a CDN are the cheapest way to reach a huge crowd.
- Recordings. A recorded stream is played as files anyway.
Live Stream plays over WebRTC only today. Its replays are recordings, played after the stream ends.
Frequently asked questions
Is WebRTC only for small audiences?
No. With media servers passing the stream on, one host reaches many viewers. A stream in Live Stream has no limit of viewers: each site streams to 50 at once unmetered, and each viewer above that takes a credit a minute (Viewers and credits).
Does a lower delay cost picture quality?
A small buffer leaves less time to ask for a lost packet again. That is why the buffer of a Live Stream player grows when its connection loses packets, and why layers let a slow connection step down to a smaller picture instead of stopping.
Do viewers need a special browser?
No. Every current browser plays WebRTC, on computers and phones.
What delay can I expect from YouTube or Facebook Live?
These platforms deliver over HLS or DASH, and some offer a low latency mode. Expect a few seconds at best, more for viewers far away or on slow connections.