Thanks for following us as we continue to dive into Fun with Container Formats. This week we’ll be jumping into MPEG Transport Stream (MPEG – TS) and Matroska. Don’t forget to read our deep dive into terminology and handling and the breakdown of MP4 and CMAF.
MPEG Transport Stream (MPEG-TS)
MPEG Transport Stream was standardized in MPEG-2 Part 1 and specifically designed for Digital Video Broadcasting (DVB) applications. Compared to its counterpart, the MPEG Program Stream, which was aimed for storing media and found its use in applications like DVD, the MPEG Transport stream is a more transport-oriented format. An MPEG Transport Stream consists of small individual packets which should be a measure to achieve resilience against and minimize implications of corruption or loss of such. Furthermore, the format includes the use of Forward Error Correction (FEC) techniques to allow correction of transmission errors at the receiver. The format was clearly designed for the use on lossy transport channels.
Muxing: ES → PES → TS
The elementary stream coming from the encoder is first turned into a packetized elementary stream (PES). The thereby added PES header, includes a stream identifier, the PES packet length, and information about media timestamps, among other things. In the next step, the PES is split up into 184-byte chunks and turned into the Transport Stream (TS) by adding a 4-byte header to each chunk. The resulting TS consists of packets with a fixed length of 188 bytes. Each TS packet’s header carries the same PID (Packet Identifier) which associates the packet with the elementary stream it originated from.
Muxing multiple Elementary Streams
One elementary stream represents either audio or video content. Given a video elementary stream, we would usually also have at least one audio elementary stream. These related ES would then be muxed into the same transport stream with there being separate PIDs for the different ES and their packets.
Muxing multiple Programs
MPEG-TS has the notion of programs. A program is basically a set of related elementary streams that belong together, e.g. video and the matching audio. A single transport stream can carry multiple programs with each being, for example, a different TV channel.
Program Association
From a low-level perspective a transport stream is just a sequence of 188 byte long TS packets. As previously mentioned, there can be many programs with each having multiple elementary streams, but a client usually is only able to present one program at a time. Therefore it must somehow know which packets to consume and which to discard upon receiving the transport stream. For this purpose there are two kinds of special packets:
- Program Association Table (PAT)
PAT packets have the reserved PID of 0 and contains the PIDs for the Program Map tables of all programs of the transport stream.
- Program Map Table (PMT)
The PMT represents a single program and contains the PIDs for all elementary streams of the program
- Inspect the TS Packet with PID 0, which contains the PAT
- Find the PMT-PID of the Program the player should play back in the PAT (in this example: 200)
- Get the TS Packet with the relevant PMT-PID, which contains the PMT (in this example: PID 200)
- The PMT contains the PID for all the media tracks, which are part of the Program to play
A client receiving the transport stream would first read the first PAT packet it receives and pick the program to be presented depending on the user’s selection. From the PAT it would get the selected program’s PMT which provides the program’s elementary streams and their PIDs. Now the client would just filter for these PIDs, each representing a separate ES of the chosen program, and consume them, i.e. demux, decode and present them to the user.
OTT-specific aspects and Conclusion
The previous explanations were very broadcast-oriented, in OTT, however, there are special considerations to be made. OTT clients have network connections that are unstable and do not have a guaranteed bandwidth which requires that only content that will be presented should be loaded. Given that a client is only able to present a single program at a time, having multiple programs in the same transport stream and loading them would be a waste of bandwidth that could be better used otherwise, e.g. for quality adaptation. So for OTT we would never have multiple programs in one transport stream. For multi-audio content similar arguments apply regarding multiplexing elementary streams, i.e. all of them should be multiplexed into their own transport stream.
MPEG-TS is still widely used for OTT, especially when targeting the Apple ecosystem.
A downside of MPEG-TS is that due to the small packet size and all the packet headers the [overhead is higher](https://www.slideshare.net/bitmovin/bitmovin-segmentlength-50056628/11) compared to fMP4.
MPEG-TS is transport-oriented and includes considerations for lossy communication channels which is not exactly a good fit for HTTP-based media delivery where transport loss is already handled by the network stack.
Debugging and Inspecting MPEG-TS
- http://www.digitalekabeltelevisie.nl/dvb_inspector/ (GUI, open source)
- http://thumb.co.il/ (GUI/Web, open source)
- http://dvbsnoop.sourceforge.net/ (CLI, open source)
- https://github.com/tsduck/tsduck (CLI, open source)
Matroska (Webm)
Overview
Matroska is a free and open-standard container format. It is based on the Extensible Binary Meta Language (EBML), which is basically XML in binary form. This fact makes the standard easy to extend. It is possible to virtually support any codec.
WebM
WebM is based on the Matroska container format. The development was mainly driven by Google to have a free and open alternative to MP4 and MPEG2-TS to be used on the web. It was also developed to support Google’s open and free codecs like: VP8, VP9 for video and Opus and Vorbis for audio. It is also possible to use WebM with DASH to stream VP9 and Opus over the web.
Debugging Matroska/Webm
The best tool to debug and view the contents of a Matroska or WebM file is mkvinfo (https://mkvtoolnix.download/). Thanks for following our Fun with Container Formats series! To learn more about container formats and handling join us at our Monthly Demo and Technical Q&A!
Resources:
Video technology guides and articles
- Back to Basics: Guide to the HTML5 Video Tag
- What is a VoD Platform? A comprehensive guide to Video on Demand (VOD)
- Video Technology [2022]: Top 5 video technology trends
- HEVC vs VP9: Modern codecs comparison
- What is the AV1 Codec?
- Video Compression: Encoding Definition and Adaptive Bitrate
- What is adaptive bitrate streaming
- MP4 vs MKV: Battle of the Video Formats
- AVOD vs SVOD; the “fall” of SVOD and Rise of AVOD & TVOD (Video Tech Trends)
- MPEG-DASH (Dynamic Adaptive Streaming over HTTP)
- Container Formats: The 4 most common container formats and why they matter to you.
- Quality of Experience (QoE) in Video Technology [2022 Guide]
