Last month more than 200 software engineers gathered at Studio Spaces in London to attend the first ever Dev Tools @Scale. The event had eight exciting presentations focused on development tools that either work at large scale or enable large scale development. Speakers from Amazon, ARM, Facebook, GitHub, Google, Microsoft, and Spotify covered many aspects of the software development cycle, starting from code search through source control and code quality to build and testing.
For a recap of the conference and the presentations, have a look at the videos below. The @Scale community is focused on bringing people together to openly discuss these challenges and collaborate on the development of new solutions. If you’re interested in joining the next event, visit the @Scale website or join the @Scale community.
One World: Resource Management at Scale
Phyllipe Medeiros, Facebook
Phyllipe talks about Project OneWorld, a resource management system at Facebook that enables end-to-end testing using a variety of devices at a large scale, covering Android emulators, Android devices, iOS simulators, and web browsers.
A Tale of Two Pizzas: Accelerating Software Delivery with AWS Developer Tools
Paul Maddox, Amazon
Paul introduces Amazon’s productivity tools for reducing software development release cycles, including tools for continuous integration and delivery workflows. He describes how Amazon tools evolved from monolithic applications to micro-services, then further to pipelines, and how this increased productivity.
Facebook Infer: Advanced Static Analysis at Speed and Scale
Dulma Churchill & Jules Villard, Facebook
Dulma and Jules talk about how to improve code quality through static analysis. They describe the Infer static analyzer, an open source tool developed at Facebook. This tool is fast enough to run as part of continuous integration and can catch problems such as null dereferences and memory leaks before code is committed.
Top Ten Worst Repositories to Host on GitHub
Carlos Martin Nieto, GitHub
Carlos describes the diverse and challenging use cases that GitHub needs to accommodate. They face complexities arising from both the size and shape of the stored data. GitHub has applied various techniques using replication and caching controlled by heuristics to host millions of Git repositories reliably.
Scaling Up Job Scheduling With a Match-making Service
Adriana Liborio, Facebook
Adriana presents Jupiter, the solution Facebook is using to schedule continuous integration jobs in a pool of thousands of servers. The requirements of the jobs vary, and so do the capabilities of the server resources. Jupiter can find and schedule jobs that meet the minimum SLA requirements of the jobs in a scalable and efficient way.
Scaling the Git Client with the Git Virtual File System
Edward Thomson, Microsoft
Edward shows the tools Microsoft developed to scale Git to hundreds of gigabytes and millions of files in a single repo, and how they used these tools to migrate the Windows team to Git. Scaling source control to such size is hard, and a large part of the success in this monorepo is due to the use of the Git Virtual File System (GVFS) implemented at Microsoft.
Scaling Android Application Testing at Spotify
Sean Kenny, Spotify
Sean describes how Spotify’s testing tools and infrastructure are keeping up with the rapid growth of Spotify’s engineering team. He shows how they improved testing quality through better reporting, enforced linters, and ways to eliminate flakiness in tests.
Cross-platform Dev Tools for Million-core Applications
Chris January, ARM
Chris shows how applications that scale to millions of cores can still be debugged and profiled in an easy-to-use way. The tools he presents enable domain experts with programming as a secondary skill to understand the performance bottlenecks of their applications without being overwhelmed by information of tens of thousands of active processes.
Searching Through Source Code at Scale
Jeroen Vaelen, Facebook
Jeroen talks about how development productivity can be boosted through investing into tools that allow fast search through source code and its history. Not only do such tools aid the exploration and understanding of large code bases, they enable a large number of additional tools such as refactorings and analysis.
Bazel: Google’s Extensible, Multi-lingual, Scalable Build System
Dmitry Lomov, Google
Dmitry walks us through how Bazel, Google’s recently open-sourced build system, works. He shows how it can produce repeatable and reproducible output in a performant way in large code bases, and how it can be extended seamlessly to different languages.
