This year’s @Scale is right around the corner — just a few weeks away. Engineers from Airbnb, Box, Facebook, GitHub, Google, Instagram, LinkedIn, Microsoft, Pinterest, Twitter, and Uber will be discussing new solutions for building at scale. Attendees will hear best practices from some of the industry’s leading engineering teams around in-house data systems, source control management, IPv6, social graph caches, streaming systems, source control management models, Heron, data analytics, data serving, mass messaging, React Native, mobile apps for unreliable networks, and much more.
Register for the 2015 @Scale conference here. The event will be at the San Jose Convention Center on September 14, from 8 a.m. to 8 p.m.
Not sure if you’re on the invite list? That’s OK! Visit this link and fill out a brief form telling us why you’d like to attend.
For a taste of what the event is like, check out this slideshow. For a deeper dive on what’s in store this year, read the summary of talks below.
Data: Summary of talks
- Heron is now the de facto stream data processing engine at Twitter, and Karthik Ramasamy will share experiences from running it in production.
- Pinterest‘s Varun Sharma will present a new in-house system his team built for serving large data sets generated through Hadoop/Hive jobs.
- David MacKenzie will share Box‘s secret sauce behind supporting its events API, which powers its desktop sync experience, with HBase to store and serve a separate message queue for each of its 30+ million users.
- Microsoft‘s Arun Jayandra will discuss the experience and learnings he and the Microsoft Office 365 team have gained over the past six months leveraging Spark Streaming and Scala jobs for Batch processing, with Cassandra being used to store raw events and serve real-time requests.
- Facebook‘s Nathan Bronson will cover the systems at Facebook that serve the social graph, which are multi-tenant. Data is sharded across many machines, but each database and cache server holds all types of data and answers queries for all products. He’ll share some of the core ideas around request queuing in TAO, the company’s cache for the social graph.
- Airbnb runs both production site and data infrastructure completely within AWS, and Paul Yang will be talking about how the company migrated to a two HDFS/Hive cluster setup and built significant tooling to keep multiple petabytes of data in sync across those clusters.
- Rene Schmidt will discuss Uber‘s internal storage system, called schemaless, which has been built and deployed to store transactional data such as trip history, but with a couple of unique twists, such as being an append-only system.
- Frances Perry will talk about how Google has evolved its earlier work on batch and streaming systems, including MapReduce, FlumeJava, and Millwheel into Dataflow, a new programming model that allows users to clearly trade off correctness, latency, and cost.
- Ostap Korkuna will dive into the evolution of Facebook‘s monitoring system and the current challenges his team faces, including anomaly detection at scale, driving data exploration, and intelligent spam fighting.
Dev Tools: Summary of talks
- Google‘s Melody Meckfessel will discuss how her team at Google writes their systems through continuous delivery, how DevOps enables them to speed up launching features, and how their engineering culture thrives.
- Facebook‘s Joe Savona, Christoph Pojer, and Nick Schrock will explore the structure of GraphQL servers, strategies for adopting them in an organization, and the client tooling unlocked from doing so. They’ll also discuss the Relay framework and an overview of the architecture, including what’s next.
- Peter Seibel will talk about lessons learned behind Twitter‘s growth from a tiny company that hosts a website built on Ruby on Rails to a slightly larger one that hosts a website built on the world’s largest Ruby on Rails app (the Monorail), and then to a company that employs more than a thousand engineers and hosts a website and mobile apps built on hundreds of JVM-based services that run in multiple data centers.
- Google‘s Rachel Potvin will outline the scale of Google’s codebase, describe the company’s custom-built monolithic source repository, and discuss the reasons behind choosing this model of source control management.
- Facebook‘s Paul Saab will discuss the growth of IPv6 on Facebook and how it will affect application developers moving forward.
- Microsoft‘s Jonathan Bergeron and Jakub Grzmiel will discuss how engineering systems that support this rapid time-to-market also enable the company’s developers to easily scale out to a multitude of experiences from mobile, Xbox, PC, and tablet for international markets, and then send back data in near real-time to allow further refinements of the shipped idea that same evening.
- GitHub‘s Ben Ogle will join Facebook‘s Ryan Bergauer and Jess Lin to talk about increased difficulties in supporting a number of engineers and a diverse selection of languages with off-the-shelf development tools and Nuclide, an IDE designed to address these problems and provide a single, scalable editing environment for mobile, web, and more.
Mobile: Summary of talks
- Twitter‘s Jess Garms will talk about how his team at Twitter builds mobile apps for unreliable networks.
- Facebook‘s Jonathan McKay and Nate Schloss will present with Owen Campbell-Moore from Google to discuss the work Facebook has done with Google, UC, and Opera by using private and public APIs to bring the benefits of native apps to everyone.
- Clement Genzmer and Greg Moeck will share some of the innovative tooling Instagram has developed to monitor CPU consumption during scrolling, larger strategies for optimizing UIs, and “tips and tricks” they’ve uncovered deep down in iOS and UIkit to harness the power of every core.
- Martin Destagnol will discuss how Box transitioned its mobile strategy from building primarily end user apps to a state where mobile SDKs and mobile platform components come first, highlighting findings and challenges along the way.
- Peter Cottle and Brian Sa will talk about how Facebook leverages the social graph to reach millions of people affected by natural disasters and deliver news of their well-being to their families and friends, and generally how personalized announcements across the web and mobile clients can be delivered quickly and efficiently.
There are a lot of ways to stay connected with @Scale. Join the @Scale community to stay up to date, or check the @Scale conference site for more updates on our speaker lineup and full agenda.
