Engineers from hundreds of companies gathered in San Jose yesterday for the annual @Scale conference, an event for engineers who work on large-scale platforms and technologies. Building applications and services that scale to millions or even billions of people presents a complex set of engineering challenges, many of them unprecedented. The @Scale community is focused on bringing people together to openly discuss these challenges and collaborate on the development of new solutions.
This year's event featured talks from more than 20 companies across four tracks: data, dev tools, mobile, and a new hot topics track focused on innovative technologies such as video and machine learning. Throughout the day, Facebook speakers unveiled some of the new tools and technologies we've built to address our own scale challenges and the performance improvements that we've achieved with them.
Zstandard
In his data track talk, Yann Collet introduced a new compression algorithm called Zstandard, which compresses data smaller and faster than today's industry norm. It also decompresses at twice the speed and scales to much higher compression ratios, which translates directly to faster data transfer and smaller storage requirements. Zstandard 1.0 is now available as both a command line tool and a library.
MyRocks
We also announced MyRocks, a new open source MySQL storage engine on top of RocksDB that works more efficiently than InnoDB on flash storage. Yoshinori Matsunobu described the MyRocks architecture and how Facebook migrated the new storage engine to its user database tier in one data center region, reducing storage requirements by 50 percent on those servers.
360 video stabilization
Last year we announced 360 videos for News Feed and have seen an increase in the range and volume of 360 content. In the Hot Topics track, Johannes Kopf presented a new stabilization technology to make shaky 360 videos smoother and ensure a high-quality, consistent viewing experience for all 360 videos across Facebook and Oculus. In his talk, he delves into the technology behind the new algorithm, an approach that combines 2D and 3D stabilization methods with a new “deformed rotation” motion model, and highlights performance improvements such as reduced bit rate for the same video quality and fast upload and playback speeds.
BrowserLab
With thousands of engineers committing hundreds of code changes per day at Facebook, even small regressions can add up quickly. In a Dev Tools talk, Jeffrey Dunn presented BrowserLab, an automated testing environment capable of detecting regressions as small as 20 ms and preventing those bugs from reaching production. He covers techniques for handling highly dynamic content, isolating server variance from client-side performance, and accounting for non-deterministic code, which together enable engineers to better understand the performance implications of their code changes and optimize end-to-end performance on the web.
Infer
Another way that Facebook engineers catch bugs early in the development process is through static analysis. With a Facebook-scale codebase, these tools must be adaptable for a range of variables. Sam Blackshear discusses Infer, Facebook's open source static analyzer for Java, C, and Objective-C, and explains how the framework was designed to make it easy to add new bug types, analyses, and languages. By doing this, the team has now begun to add support for C++.
To stay updated about future @Scale events, please join our community or visit https://atscaleconference.com/ for more information.
