VVC – the latest evolution for modern codecs
Versatile Video Coding (h.266/VVC) is the newest block-based hybrid codec from the Joint Video Experts Team (JVET), a group comprised of MPEG and ISO/ITU members such as Bitmovin and Fraunhofer HHI, and promises to vastly improve the compression capabilities of workflows for any organization within the streaming industry, including but not limited to, OTT, VR, AR, and many other providers. As fellow members of MPEG, the Bitmovin encoding team was eager to test the capabilities of the newest codec and the potential improvements it offered over its predecessor h.265/HEVC. The ultimate goal of the project was to determine the performance parameters of the VVC codec and the subjective visual quality enhancements that ensue. While Fraunhofer HHI claimed that the VVC codec promises to improve visual quality and reduce bitrate expenditure by around 50% over HEVC, we wanted to prove the validity of the statement.
“Overall, H.266/VVC provides efficient transmission and storage of all video resolutions from SD to HD up to 4K and 8K, while supporting high dynamic range video and omnidirectional 360° video.” – Fraunhofer HHI
The end goal of our research was to implement the VVC distribution process into Bitmovin’s standard encoding process, as illustrated below:
Developing a VVC Encoding Process
To kick-off our experiment, we added the VVC Test Model (VTM) encoder library into our encoder with a flexible interface. However, some critical changes were needed in the Bitmovin API to enable VVC encoding.
Implementing the VVC codec
The first step to enable VVC encoding is to add VVC as a new video codec to our API. Since we haven’t established a “real” encoder yet, there are limited settings, parameters, and inputs that we can use. For the first set of tests we used the following: width, height, bitrate, pixelFormat, rate, and Constant Rate Factor (CRF); as seen below:
There are additional input values necessary for a real VVC encoding but didn’t apply for our test. These are colorConfig, sampleAspectRatio, encodingMode, preset, and profile. Any other specialized settings will depend on the encoder implementation that we use when live. At the time of our test, VVenC and other encoder implementations, like x266 were not yet available and thus the settings are unknown. Now that the VVC codec parameters have been established we needed to actually implement them into the Bitmovin encoder.
Testing VVC in Bitmovin’s Encoder
Thanks to our flexible interface, Lead Encoding Engineer, Christian Feldmann, had already added the library interface to the open-sourced reference encoder (VTM) and also added a patch to our internal encoder, which pushes frames to the VTM encoder and retrieves any data from it. Then, we implemented a new Bitmovin encoder Dependency that contains VTM with h.266 support.
Lastly, we created a VideoService integration test that would prove that the VTM integration works. However, as expected of a brand new codec implementation – most of the initial tests failed.
Trial, error, and repeat: VVC encoding initial failures
Although expected, we were disappointed to find that running multiple renditions using the h.266 codec doesn’t work as the VTM implementation uses global variables. Thus we were not able to launch multiple encoding tasks on the same “machine” (respectively within the same docker container). Another issue that we encountered is that only the 1080p output format worked as a result of the encode. While promising, the VVC codec and encoding are intended to work for nearly any output format, and a single output is clearly not the intended result. To resolve these issues we implemented a few minor patches within our API.
Resolving the VVC encoding failures
Fortunately for our experiment, a simple patch to the encoder, as well as the implementation of the libVTM worked almost out-of-the-box. The remaining issues were two minor bugs that required the connection of the CRF value of our API to the baseQP value in libVTM.
Now that the encoding issues were resolved, we were able to run the first objective visual quality evaluations using the Peak Signal-to-Noise Ratio (PSNR). As a baseline, we ran four different CRF values at 1080p resolution to calculate BD-rate curves in comparison to h.265/HEVC. The result was h266 and h265 output files with the same four CRFs that allowed the first quality comparison.
How did VVC stack up?
Once our initial implementation tests were complete, we set up a complete end-to-end encoding test with a single asset, Mango’s open-source project Tears of Steel.
Side by side comparison video of encoded asset VVC vs HEVC
Given the limitations of the encoder implementations, VVC’s performance was as promised (and expected), offering ~45% percent lower bitrate compared to HEVC. As visualized in the graph below, our results are nearly identical to that of the official measurements provided by JVET.
Although there are significant visual quality improvements and at lower bitrate expenditure, the current encoding results came at an immense computing and time cost. In our initial tests, the encoding time for a 4-second video ranged between 2.5-6.4 hrs (varied based on output parameters).
What does this mean moving forward?
Our initial tests of the VVC codec were run with the bare bones VTM encoder implementation without any adjustments, optimizations, or improvements. As the industry continues to develop the back-end tech to support h.266/VVC, we can expect that encoding times and computing power will decrease significantly in the very near future. Since the Bitmovin encoding team ran this first test, our academic and research arms ran additional tests at both the Alpen Adria University in Klagenfurt, as well as at the 132nd MPEG meeting. The next set of verification tests extended VVC’s capabilities to successfully encode ultra-high definition (UHD) content with standard dynamic range so that it can be used in newer streaming and broadcast television applications. The latest tests were run using the recently released open-source encoder implementation of VVC (VVenC), which displayed an additional 10% bitrate savings over the original VTM implementation. The Bitmovin encoding team plans to continue testing the VVC codec and adding new encoder implementations into our workflows – with expectations of officially launching VVC support sometime in the near future.
Video technology guides and articles
- Back to Basics: Guide to the HTML5 Video Tag
- What is a VoD Platform?A comprehensive guide to Video on Demand (VOD)
- Video Technology [2022]: Top 5 video technology trends
- HEVC vs VP9: Modern codecs comparison
- What is the AV1 Codec?
- Video Compression: Encoding Definition and Adaptive Bitrate
- What is adaptive bitrate streaming
- MP4 vs MKV: Battle of the Video Formats
- AVOD vs SVOD; the “fall” of SVOD and Rise of AVOD & TVOD (Video Tech Trends)
- MPEG-DASH (Dynamic Adaptive Streaming over HTTP)
- Container Formats: The 4 most common container formats and why they matter to you.
- Quality of Experience (QoE) in Video Technology [2022 Guide]
