At Meta, we support real-time communication (RTC) for billions of people through our apps, including Messenger, Instagram, and WhatsApp. We’ve seen significant benefits by adopting the AV1 codec fo…
Overview
The article discusses how Meta is enhancing real-time communication (RTC) video quality across its apps by adopting the AV1 codec and HD capabilities. It outlines the challenges faced in low and high bandwidth scenarios and the strategies implemented to improve video quality while managing CPU and battery usage.
What You'll Learn
1
How to improve video quality for low-bandwidth RTC users using AV1 codec
2
Why AV1 codec is preferred over H.264 for mobile RTC applications
3
When to enable HD video settings in RTC applications based on network conditions
Prerequisites & Requirements
- Understanding of video codecs and real-time communication principles
- Familiarity with WebRTC and video encoding technologies(optional)
Key Questions Answered
What are the benefits of using AV1 codec for mobile RTC?
The AV1 codec offers significant improvements in video quality at lower bitrates compared to older codecs like H.264. This allows for a better user experience, especially for those on bandwidth-constrained networks, making it ideal for real-time communication applications.
How does Meta improve video quality for low-bandwidth users?
Meta enhances video quality for low-bandwidth users by implementing AV1 codec, custom video scalers, and region-of-interest encoding. These strategies allow for better allocation of bitrate to important areas of the video, improving overall quality even in challenging network conditions.
What challenges does AV1 codec present for mobile devices?
While AV1 provides better coding efficiency, it also leads to increased CPU and battery usage, which can affect the performance of mobile devices during real-time communication. This necessitates careful management of resources to ensure a good user experience.
When should HD video settings be enabled in RTC applications?
HD video settings should be enabled based on machine learning predictions of network conditions from previous calls. This ensures that only users with sufficient bandwidth and device capabilities receive high-definition video, optimizing resource usage.
Key Statistics & Figures
Peak Signal-to-Noise Ratio (PSNR) improvement
0.75 dB
This improvement was observed on average when using custom video scalers compared to default scalers.
Battery usage increase due to AV1 codec
5-6%
This increase was noted during public tests, highlighting the trade-off between video quality and battery consumption.
Low-end network bandwidth threshold
300 Kbps
Bandwidth below this threshold is considered low-end for video calls, with many operating at even lower rates.
Technologies & Tools
Codec
Av1
Used for encoding video in real-time communication applications to improve quality at lower bitrates.
Framework
Webrtc
Utilized for real-time communication in Meta's applications.
Key Actionable Insights
1Implement AV1 codec in your RTC applications to significantly enhance video quality at lower bitrates.This is particularly beneficial for users on low-bandwidth networks, allowing for a better user experience without requiring excessive data usage.
2Utilize custom video scalers to improve the quality of downscaled video in real-time communications.By leveraging in-house scalers, you can achieve better quality outcomes compared to standard scalers, especially when operating at low bitrates.
3Incorporate region-of-interest encoding to prioritize important areas of the video during encoding.This technique allows for better utilization of available bandwidth, ensuring that critical visual information, such as a speaker's face, is encoded with higher quality.
Common Pitfalls
1
Overlooking the increased CPU and battery usage when implementing AV1 codec.
Many developers may focus solely on the quality improvements without considering the trade-offs in resource consumption, which can lead to poor user experiences on mobile devices.
2
Failing to properly manage codec switching during RTC calls.
Not implementing seamless codec negotiation can result in noticeable glitches or pauses during video calls, negatively impacting user experience.
Related Concepts
Video Encoding Techniques
Real-time Communication Best Practices
Network Bandwidth Management Strategies