NVIDIA and Netflix's VMAF-CUDA: Accelerating Video Quality Assessment
The growing demand for streaming services has posed a significant challenge for the industry: delivering a high-quality viewing experience without compromising bandwidth. To tackle this issue, advanced video assessment metrics have been developed, among which VMAF (Video Multi-Method Assessment Fusion) stands out, a standard created by Netflix that accurately approximates human visual perception.
However, the traditional calculation of VMAF has been a resource and time-consuming process on CPUs. In response to this limitation, NVIDIA and Netflix have joined forces to present VMAF-CUDA, a solution that introduces GPU acceleration in VMAF 3.0 and FFmpeg v6.1.
What is VMAF and Why is Its Assessment Complex?
Unlike conventional metrics such as PSNR (Peak Signal-to-Noise Ratio), which focuses on pixel difference detection, and SSIM (Structural Similarity Index), which emphasizes brightness and contrast, VMAF integrates a human vision modeling approach with machine learning techniques.
The evaluation of a coded image compared to its reference involves analyzing three fundamental components:
Read also
- VIF (Visual Information Fidelity): which measures the perceived loss of information versus the original content.
- ADM (Additive Distortion Measurement): which examines structural changes and texture degradation, being particularly sensitive to distortions like noise.
- Motion metrics: which evaluate the representation of motion in dynamic scenes over time.
A regressor based on Support Vector Machines (SVM) combines these components to generate the final VMAF score.
Technological Evolution: CPU vs. GPU (CUDA)
The main difference between the traditional CPU implementation and the new GPU-based solution lies in how calculations are processed.
CPU Processing
In the CPU-based approach, the VMAF calculations distribute feature extraction across multiple threads in parallel. However, the processing time per frame is limited by the slowest component, usually the VIF. Additionally, the motion-related metrics have temporal dependencies between frames that complicate parallelization.
Read also
GPU Processing with VMAF-CUDA
On the other hand, VMAF-CUDA allocates 100% of the GPU's computing power to each metric sequentially, resulting in a significant reduction in the latency of each calculation.
No PCIe bottlenecks: The integration of VMAF-CUDA into the workflow with hardware such as NVDEC (decoding) and NVENC (encoding) allows both the reference frame and the encoded frame to remain in the GPU's VRAM. This eliminates costly data transfers that typically occur between the PCIe bus and the CPU's RAM.
Performance: A Results Analysis
Tests conducted on a compute node with Dual Intel Xeon 8480 processors compared to a single NVIDIA L4 GPU have shown impressive results:
1. Frame Latency
- At 4K resolution, VMAF-CUDA achieved frame scoring access 36.9 times faster than the CPU method.
- At 1080p resolution, latency was reduced by 26.1 times.
2. Performance in FFmpeg (FPS)
Analyzing individual video streams with the libvmaf_cuda filter in FFmpeg:
- At 1080p, the NVIDIA L4 GPU reached 775 FPS, compared to 176 FPS from the dual Xeon server (4.4 times faster).
- At 4K, the GPU rendered at 178 FPS versus 64 FPS from the CPU (2.8 times faster).
Cost-Benefit Evaluation in Data Centers
For large-scale streaming platforms, server density and total cost of ownership (TCO) are crucial factors to consider. The comparison between a traditional 2U server and one optimized with 8 NVIDIA L4 cards shows the following:
Performance/ Cost Metric CPU Server (2U Dual Intel Xeon Platinum 8480) GPU Server (2U with 8x NVIDIA L4)
| Total Throughput at 4K | 235 FPS | 1,424 FPS (~6 times higher)
| Time to process 1,000 hours of 4K video | 127.6 hours | 21.0 hours
| Estimated cost per 1,000 hours (4K at 30 FPS) | $97 USD | $24 USD
This optimization allows for direct computing cost savings of up to 75%, while maintaining the same level of power consumption.
Industry Implementation: Highlighted Cases
Snap Inc.
Snap uses VMAF-CUDA in its quality assessment processes for Snapchat Memories. Thanks to this technology, their reduction in computing costs has allowed them to evaluate quality after each transcoding and automatically adjust encoding parameters, achieving an improvement of up to 2.5 times in processing times using AWS cloud instances with NVIDIA T4 GPUs.
V-Nova
V-Nova has implemented VMAF-CUDA to accelerate offline metric evaluations in its quality control infrastructure, increasing speed by more than 2 times. Additionally, they are studying the calculation of VMAF in real-time within the decision loop of the LCEVC (MPEG-5 Part 2) encoding standard.
Conclusion and Availability
The advent of VMAF-CUDA has transformed a quality assessment tool previously reliant on CPU clusters into an agile, cost-effective, and scalable process. By asynchronously integrating with NVDEC and NVENC engines, NVIDIA GPUs allow for real-time video quality monitoring without disrupting transcoding activities.
VMAF-CUDA acceleration is publicly available in the libvmaf (v3.0+) and FFmpeg (v6.1+) repositories, easily implemented through Docker containers ready for production environments.
To learn more about innovations in video technology and other related topics, you are invited to continue exploring this blog.
