Engineering at Meta
type: channel
description: Engineering at Meta Blog
date: 2025-12-19T17:16:56Z
update period: hourly
update frequency: 1
update base: 2000-01-01T12:00+00:00
generator agent: https://wordpress.org/
items: DrP: Meta’s Root Cause Analysis Platform at Scale
items: How We Built Meta Ray-Ban Display: From Zero to Polish
items: How AI Is Transforming the Adoption of Secure-by-Default Mobile Frameworks
items: Zoomer: Powering AI Performance at Meta’s Scale Through Intelligent Debugging and Optimization
items: Key Transparency Comes to Messenger
items: Efficient Optimization With Ax, an Open Platform for Adaptive Experimentation
items: Announcing the Completion of the Core 2Africa System: Building the Future of Connectivity Together
items: Enhancing HDR on Instagram for iOS With Dolby Vision
items: Open Source Is Good for the Environment

DrP: Meta’s Root Cause Analysis Platform at Scale
type: item
date: 2025-12-19T17:35:13Z
subject: Data Infrastructure
subject: ML Applications
description: Incident investigation can be a daunting task in today’s digital landscape, where large-scale systems comprise numerous interconnected components and dependencies DrP is a root cause analysis (RCA) platform, designed by Meta, to programmatically automate the investigation process, significantly reducing the mean time to resolve (MTTR) for incidents and alleviating on-call toil Today, DrP is used [...] Read More... The post DrP: Meta’s Root Cause Analysis Platform at Scale appeared first on Engineering at Meta.
encoded: Incident investigation can be a daunting task in today’s digital landscape, where large-scale systems comprise numerous interconnected components and dependencies DrP is a root cause analysis (RCA) platform, designed by Meta, to programmatically automate the investigation process, significantly reducing the mean time to resolve (MTTR) for incidents and alleviating on-call toil Today, DrP is used by over 300 teams at Meta, running 50,000 analyses daily, and has been effective in reducing MTTR by 20-80% By understanding DrP and its capabilities, we can unlock new possibilities for efficient incident resolution and improved system reliability. What It Is DrP is an end-to-end platform that automates the investigation process for large-scale systems. It addresses the inefficiencies of manual investigations, which often rely on outdated playbooks and ad-hoc scripts. These traditional methods can lead to prolonged downtimes and increased on-call toil as engineers spend countless hours triaging and debugging incidents. DrP offers a comprehensive solution by providing an expressive and flexible SDK to author investigation playbooks, known as analyzers. These analyzers are executed by a scalable backend system, which integrates seamlessly with mainstream workflows such as alerts and incident management tools. Additionally, DrP includes a post-processing system to automate actions based on investigation results, such as mitigation steps. <img fetchpriority="high" decoding="async" class="alignnone size-full wp-image-23498" src="https://engineering.fb.com/wp-content/uploads/2025/12/Meta-DrP-image-1.png" alt="" width="1999" height="1125" srcset="https://engineering.fb.com/wp-content/uploads/2025/12/Meta-DrP-image-1.png 1999w, https://engineering.fb.com/wp-content/uploads/2025/12/Meta-DrP-image-1.png?resize=580,326 580w, https://engineering.fb.com/wp-content/uploads/2025/12/Meta-DrP-image-1.png?resize=916,516 916w, https://engineering.fb.com/wp-content/uploads/2025/12/Meta-DrP-ima…

How We Built Meta Ray-Ban Display: From Zero to Polish
type: item
date: 2025-12-17T14:00:17Z
subject: Culture
subject: Virtual Reality
subject: Meta Tech Podcast
description: We’re going behind the scenes of the Meta Ray-Ban Display, Meta’s most advanced AI glasses yet. In a previous episode we met the team behind the Meta Neural Band, the EMG wristband packaged with the Ray-Ban Display. Now we’re delving into the glasses themselves. Kenan and Emanuel, from Meta’s Wearables org, join Pascal Hartig on [...] Read More... The post How We Built Meta Ray-Ban Display: From Zero to Polish appeared first on Engineering at Meta.
encoded: We’re going behind the scenes of the Meta Ray-Ban Display, Meta’s most advanced AI glasses yet. In a previous episode we met the team behind the Meta Neural Band, the EMG wristband packaged with the Ray-Ban Display. Now we’re delving into the glasses themselves. Kenan and Emanuel, from Meta’s Wearables org, join Pascal Hartig on the Meta Tech Podcast to talk about all the unique challenges of designing game-changing wearable technology, from the unique display technology to emerging UI patterns for display glasses. You’ll also learn what particle physics and hardware design have in common and how to celebrate even the incremental wins in a fast-moving culture. Download or listen to the episode below: You can also find the episode wherever you get your podcasts, including: Spotify Apple Podcasts Pocket Casts The Meta Tech Podcast is a podcast, brought to you by Meta, where we highlight the work Meta’s engineers are doing at every level – from low-level frameworks to end-user features. Send us feedback on Instagram, Threads, or X. And if you’re interested in learning more about career opportunities at Meta visit the Meta Careers page. The post How We Built Meta Ray-Ban Display: From Zero to Polish appeared first on Engineering at Meta.

How AI Is Transforming the Adoption of Secure-by-Default Mobile Frameworks
type: item
date: 2025-12-15T17:00:25Z
subject: Android
subject: iOS
subject: ML Applications
subject: Security & Privacy
description: Meta’s secure-by-default frameworks wrap potentially unsafe OS and third-party functions, making security the default while preserving developer speed and usability. These frameworks are designed to closely mirror existing APIs, rely on public and stable interfaces, and maximize developer adoption by minimizing friction and complexity. Generative AI and automation accelerate the adoption of secure frameworks at [...] Read More... The post How AI Is Transforming the Adoption of Secure-by-Default Mobile Frameworks appeared first on Engineering at Meta.
encoded: Meta’s secure-by-default frameworks wrap potentially unsafe OS and third-party functions, making security the default while preserving developer speed and usability. These frameworks are designed to closely mirror existing APIs, rely on public and stable interfaces, and maximize developer adoption by minimizing friction and complexity. Generative AI and automation accelerate the adoption of secure frameworks at scale, enabling consistent security enforcement and efficient migration across Meta’s vast codebase. Sometimes functions within operating systems or provided by third parties come with a risk of misuse that could compromise security. To mitigate this, we wrap or replace these functions using our own secure-by-default frameworks. These frameworks play an important role in helping our security and software engineers maintain and improve the security of our codebases while maintaining developer speed. But implementing these frameworks comes with practical challenges, like design tradeoffs. Building a secure framework on top of Android APIs, for example, requires a thoughtful balance between security, usability, and maintainability. With the emergence of AI-driven tools and automation we can scale the adoption of these frameworks across Meta’s large codebase. AI can assist in identifying insecure usage patterns, suggesting or automatically applying secure framework replacements and continuously monitoring compliance. This not only accelerates migration but also ensures consistent security enforcement at scale. Together, these strategies empower our development teams to ship well-secured software efficiently, safeguarding user data and trust while maintaining high developer productivity across Meta’s vast ecosystem. How We Design Secure-by-Default Frameworks at Meta Designing secure-by-default frameworks for use by a large number of developers shipping vastly different features across multiple apps is an interesting challenge. There are a lot of competing concerns…

Zoomer: Powering AI Performance at Meta’s Scale Through Intelligent Debugging and Optimization
type: item
date: 2025-11-21T21:00:15Z
subject: Data Center Engineering
subject: Data Infrastructure
subject: ML Applications
description: We’re introducing Zoomer, Meta’s comprehensive, automated debugging and optimization platform for AI. Zoomer works across all of our training and inference workloads at Meta and provides deep performance insights that enable energy savings, workflow acceleration, and efficiency gains in our AI infrastructure. Zoomer has delivered training time reductions, and significant QPS improvements, making it the [...] Read More... The post Zoomer: Powering AI Performance at Meta’s Scale Through Intelligent Debugging and Optimization appeared first on Engineering at Meta.
encoded: We’re introducing Zoomer, Meta’s comprehensive, automated debugging and optimization platform for AI. Zoomer works across all of our training and inference workloads at Meta and provides deep performance insights that enable energy savings, workflow acceleration, and efficiency gains in our AI infrastructure. Zoomer has delivered training time reductions, and significant QPS improvements, making it the de-facto tool for AI performance optimization across Meta’s entire AI infrastructure. At the scale that Meta’s AI infrastructure operates, poor performance debugging can lead to massive energy inefficiency, increased operational costs, and suboptimal hardware utilization across hundreds of thousands of GPUs. The fundamental challenge is achieving maximum computational efficiency while minimizing waste. Every percentage point of utilization improvement translates to significant capacity gains that can be redirected to innovation and growth. Zoomer is Meta’s automated, one-stop-shop platform for performance profiling, debugging, analysis, and optimization of AI training and inference workloads. Since its inception, Zoomer has become the de-facto tool across Meta for GPU workload optimization, generating tens of thousands of profiling reports daily for teams across all of our apps. Why Debugging Performance Matters Our AI infrastructure supports large-scale and advanced workloads across a global fleet of GPU clusters, continually evolving to meet the growing scale and complexity of generative AI. At the training level it supports a diverse range of workloads, including powering models for ads ranking, content recommendations, and GenAI features. At the inference level, we serve hundreds of trillions of AI model executions per day. Operating at this scale means putting a high priority on eliminating GPU underutilization. Training inefficiencies delay model iterations and product launches, while inference bottlenecks limit our ability to serve user requests at scale. Remov…

Key Transparency Comes to Messenger
type: item
date: 2025-11-20T17:00:21Z
subject: Security & Privacy
description: We’re excited to share another advancement in the security of your conversations on Messenger: the launch of key transparency verification for end-to-end encrypted chats. This new feature enables an additional level of assurance that only you — and the people you’re communicating with — can see or listen to what is sent, and that no [...] Read More... The post Key Transparency Comes to Messenger appeared first on Engineering at Meta.
encoded: We’re excited to share another advancement in the security of your conversations on Messenger: the launch of key transparency verification for end-to-end encrypted chats. This new feature enables an additional level of assurance that only you — and the people you’re communicating with — can see or listen to what is sent, and that no one else, not even Meta, can do so. End-to-end encryption on Messenger already ensures that the content of your direct messages and calls are protected from the moment they leave your device to the moment they reach the receiver’s device. As part of our end-to-end encrypted chat platform, we believe it’s also important that anyone can verify that the public keys (used by the sender’s device for encrypting each message) belong to the intended recipients and haven’t been tampered with. This launch builds upon the valuable work and experiences shared by others in the industry. WhatsApp’s implementation of key transparency in 2023 demonstrated the feasibility of this technology for large-scale encrypted messaging. We’ve extended these pioneering efforts in our Messenger implementation to deliver a robust and reliable solution with similar security properties. What Is Key Transparency? Key transparency provides messaging users with a verifiable and auditable record of public keys. It allows them to confirm that their conversations are indeed encrypted with the correct keys for their contacts, and that these keys haven’t been maliciously swapped by a compromised server. This means you can be more confident that your messages are only accessible to the people you intend to communicate with. <img loading="lazy" decoding="async" class="alignnone size-full wp-image-23383" src="https://engineering.fb.com/wp-content/uploads/2025/11/Meta-Key-Transparency-Messenger.png" alt="" width="1999" height="1646" srcset="https://engineering.fb.com/wp-content/uploads/2025/11/Meta-Key-Transparency-Messenger.png 1999w, https://engineering.fb.com/wp-content/uploads…

Efficient Optimization With Ax, an Open Platform for Adaptive Experimentation
type: item
date: 2025-11-18T17:00:24Z
subject: Open Source
description: We’ve released Ax 1.0, an open-source platform that uses machine learning to automatically guide complex, resource-intensive experimentation. Ax is used at scale across Meta to improve AI models, tune production infrastructure, and accelerate advances in ML and even hardware design. Our accompanying paper, “Ax: A Platform for Adaptive Experimentation” explains Ax’s architecture, methodology, and how it [...] Read More... The post Efficient Optimization With Ax, an Open Platform for Adaptive Experimentation appeared first on Engineering at Meta.
encoded: We’ve released Ax 1.0, an open-source platform that uses machine learning to automatically guide complex, resource-intensive experimentation. Ax is used at scale across Meta to improve AI models, tune production infrastructure, and accelerate advances in ML and even hardware design. Our accompanying paper, “Ax: A Platform for Adaptive Experimentation” explains Ax’s architecture, methodology, and how it compares to other state-of-the-art black-box optimization libraries. How can researchers effectively understand and optimize AI models or systems that have a vast number of possible configurations? This is a challenge that is particularly prevalent in domains characterized by complex, interacting systems, such as modern AI development and deployment. Optimizing under these settings demands experimentation, and efficiency is of the utmost importance when evaluating a single configuration is extremely resource- and/or time-intensive. Adaptive experimentation offers a solution to this problem by actively proposing new configurations for sequential evaluation, leveraging insights gained from previous evaluations This year, we released version 1.0 of Ax, an open source adaptive experimentation platform that leverages machine learning to guide and automate the experimentation process. Ax employs Bayesian optimization to enable researchers and developers to conduct efficient experiments, identifying optimal configurations to optimize their systems and processes. In conjunction with this major release, we published a paper titled, “Ax: A Platform for Adaptive Experimentation” that explores Ax’s core architecture, provides a deeper explanation of the methodology powering the optimization, and compares Ax’s performance against other black-box optimization libraries. Ax has been successfully applied across various disciplines at Meta, including: Traditional machine learning tasks, such as hyperparameter optimization and architecture search. Addressing key challenges in GenAI, in…

Announcing the Completion of the Core 2Africa System: Building the Future of Connectivity Together
type: item
date: 2025-11-18T06:00:58Z
subject: Connectivity
description: Connecting Africa and the World We’re excited to share the completion of the core 2Africa infrastructure, the world’s longest open access subsea cable system. 2Africa is a landmark subsea cable system that sets a new standard for global connectivity. This project is the result of years of collaboration, innovation, and a shared vision to connect [...] Read More... The post Announcing the Completion of the Core 2Africa System: Building the Future of Connectivity Together appeared first on Engineering at Meta.
encoded: Connecting Africa and the World We’re excited to share the completion of the core 2Africa infrastructure, the world’s longest open access subsea cable system. 2Africa is a landmark subsea cable system that sets a new standard for global connectivity. This project is the result of years of collaboration, innovation, and a shared vision to connect communities, accelerate economic growth, and enable transformative digital experiences across Africa and beyond. https://engineering.fb.com/wp-content/uploads/2025/11/2Africa_Impact_Captions_1920x1080_Stereo_MPEG-4-1.mp4 Unprecedented Scale and Reach 2Africa is the first cable to connect East and West Africa in a continuous system and link Africa to the Middle East, South Asia, and Europe. With a current reach of 33 countries and still counting, we’re enabling connectivity for 3 billion people across Africa, Europe, and Asia – more than 30% of the world’s population. This scale is unprecedented and we are proud to have partnered with stakeholders across the ecosystem to deliver new levels of connectivity at such scale. <img loading="lazy" decoding="async" class="size-full wp-image-23340" src="https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png" alt="" width="1920" height="1080" srcset="https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png 1920w, https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png?resize=580,326 580w, https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png?resize=916,515 916w, https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png?resize=768,432 768w, https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png?resize=1024,576 1024w, https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png?resize=1536,864 1536w, https://engineering.fb.com/wp-content/uploads/2025/11/2Africa-2025_1117-Map.png?resize=96,54 96w, https://engineering.fb.com/wp-c…

Enhancing HDR on Instagram for iOS With Dolby Vision
type: item
date: 2025-11-17T17:30:25Z
subject: iOS
subject: Video Engineering
subject: Instagram
description: We’re sharing how we’ve enabled Dolby Vision and ambient viewing environment (amve) on the Instagram iOS app to enhance the video viewing experience. HDR videos created on iPhones contain unique Dolby Vision and amve metadata that we needed to support end-to-end Instagram for iOS is now the first Meta app to support Dolby Vision video, [...] Read More... The post Enhancing HDR on Instagram for iOS With Dolby Vision appeared first on Engineering at Meta.
encoded: We’re sharing how we’ve enabled Dolby Vision and ambient viewing environment (amve) on the Instagram iOS app to enhance the video viewing experience. HDR videos created on iPhones contain unique Dolby Vision and amve metadata that we needed to support end-to-end Instagram for iOS is now the first Meta app to support Dolby Vision video, with more support coming across all of Meta’s apps coming in the future. Every iPhone-produced HDR video encoding includes two additional pieces of metadata that help ensure the picture is consistent between different displays and viewing conditions: Ambient viewing environment (amve), which provides the characteristics of the nominal ambient viewing environment for displaying associated video content. This information enables the final device to adjust the rendering of the video if the actual ambient viewing conditions differ from those for which it was encoded. Dolby Vision, which enhances color, brightness, and contrast to better match the video to the capabilities of the display. While the Instagram and Facebook iOS apps have supported high dynamic range (HDR) video since 2022, our initial rollout of HDR didn’t support Dolby Vision or amve delivery and playback. Our derived encodings were done with FFmpeg, which has traditionally lacked support for Dolby Vision and amve. Since our tooling was discarding this metadata, it meant that pictures were not entirely representative of the way they were meant to be viewed – something that was particularly noticeable at low screen brightness levels. Now, after hearing feedback from people using our iOS apps, we’ve worked with our partners to preserve the iOS-produced amve and Dolby Vision metadata from end-to-end and significantly enhanced the HDR viewing experience on iOS devices. How Meta Processes Video It may first be helpful to give some background on the lifecycle of a video at Meta. The majority of videos uploaded through our apps go through three main stages: 1. Client Processing In …

Open Source Is Good for the Environment
type: item
date: 2025-11-14T20:54:14Z
subject: Data Center Engineering
subject: ML Applications
subject: Open Source
subject: Production Engineering
subject: Meta Tech Podcast
description: Most people have heard of open-source software. But have you heard about open hardware? And did you know open source can have a positive impact on the environment? On this episode of the Meta Tech Podcast, Pascal Hartig sits down with Dharmesh and Lisa to talk about all things open hardware, and Meta’s biggest announcements [...] Read More... The post Open Source Is Good for the Environment appeared first on Engineering at Meta.
encoded: Most people have heard of open-source software. But have you heard about open hardware? And did you know open source can have a positive impact on the environment? On this episode of the Meta Tech Podcast, Pascal Hartig sits down with Dharmesh and Lisa to talk about all things open hardware, and Meta’s biggest announcements from the 2025 Open Compute Project (OCP) Summit – including a new open methodology for leveraging AI to understand Scope 3 emissions. Learn about the history of OCP and its growth into an organization with more than 400 companies contributing to it. You’ll also hear how AI and open hardware are helping Meta push to achieve net zero emissions in 2030, including how AI is being used to develop new concrete mixes for data center construction. Download or listen to the episode below: You can also find the episode wherever you get your podcasts, including: Spotify Apple Podcasts Pocket Casts The Meta Tech Podcast is a podcast, brought to you by Meta, where we highlight the work Meta’s engineers are doing at every level – from low-level frameworks to end-user features. Send us feedback on Instagram, Threads, or X. And if you’re interested in learning more about career opportunities at Meta visit the Meta Careers page. The post Open Source Is Good for the Environment appeared first on Engineering at Meta.