Accelerating on-device ML on Meta’s family of apps with ExecuTorch

ExecuTorch is the PyTorch inference framework for edge devices developed by Meta with support from industry leaders like Arm, Apple, and Qualcomm.  Running machine learning (ML) models on-device is…

Meta logo
Meta
•
PyTorch Edge Team in collaboration with Family of Apps
•5 min read•intermediate•
--
•View Original
PyTorch

Overview

The article discusses ExecuTorch, Meta's PyTorch inference framework for edge devices, which enhances on-device machine learning (ML) across its family of apps. It highlights improvements in performance, privacy, and latency, showcasing specific applications in Instagram, WhatsApp, Messenger, and Facebook.

What You'll Learn

1

How to implement on-device ML models using ExecuTorch

2

Why on-device ML improves user privacy and latency

3

When to migrate existing ML models to ExecuTorch for better performance

Key Questions Answered

How does ExecuTorch enhance on-device ML for Meta's apps?
ExecuTorch enhances on-device ML by providing a compact and efficient framework that improves model performance, reduces latency, and maintains user privacy. It allows for the deployment of various ML models directly on devices, enabling features like Instagram's Cutouts and WhatsApp's bandwidth estimation.
What are the benefits of using ExecuTorch in Meta's applications?
The benefits of using ExecuTorch include improved performance and efficiency of ML models, enhanced user experiences, and the ability to keep sensitive data on users' devices, thereby ensuring privacy. This results in faster feature implementations and better overall app reliability.
What specific ML features have been implemented using ExecuTorch?
Specific ML features implemented using ExecuTorch include Instagram's Cutouts for creating personalized stickers, WhatsApp's bandwidth estimation for optimizing video calls, and Messenger's language identification for enhancing communication. These features leverage on-device processing for improved performance.
How does ExecuTorch support end-to-end encryption on Messenger?
ExecuTorch supports end-to-end encryption on Messenger by allowing models to run on-device, which keeps data transfers encrypted and secure. This migration ensures that sensitive user data remains private while still enabling advanced features like language identification.

Key Statistics & Figures

Daily Active Users (DAU) increase for Instagram's Cutouts feature
Significant increase
This was achieved by migrating the Cutouts feature to ExecuTorch, resulting in faster performance.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

ML Framework
Executorch
Used for on-device inference on mobile and edge devices.
ML Framework
Pytorch
ExecuTorch is built on PyTorch 2.x technologies for model conversion and deployment.

Key Actionable Insights

1
Adopting ExecuTorch can significantly enhance the performance of on-device ML models.
This is particularly relevant for applications that require real-time processing and low latency, such as video calls and image processing features.
2
Migrating existing ML models to ExecuTorch can improve user privacy by keeping data on-device.
This is crucial for applications that handle sensitive user information, as it reduces the risk of data breaches and enhances user trust.
3
Utilizing ExecuTorch can streamline the research-to-production path for ML models.
This allows developers to deploy features more rapidly, which is essential in the competitive landscape of mobile applications.

Common Pitfalls

1
Failing to optimize ML models for on-device deployment can lead to performance issues.
This often happens when developers do not consider the limitations of mobile hardware, leading to slow inference times and poor user experiences.

Related Concepts

On-device Machine Learning
Edge Computing
Privacy In AI Applications
Real-time Data Processing