The first six months of 2014 have been very busy for our open source program. In the spirit of the FIFA World Cup, we thought it was time for a half-time review of some of the highlights so far.
Since January, we’ve launched 64 new projects – an average of more than two each week – bringing our active portfolio to exactly 200 repositories across our Facebook, Instagram, and Parse GitHub accounts.
Overall activity has been stronger than ever. There have been 13,000 total commits to these repos (an increase of 45% from the second half of 2013), and the overall size of our open source code base has risen 51% – in six months – to 9.9 million lines of code.
We’ve also been thrilled to see community participation in these projects increase: collectively, our projects have been forked 20,000 times and have 95,000 followers between them. A quarter of of all commits are regularly from community contributors (up from 15% last year), and in fact some of our major projects – notably React and Pop – have even seen the number of external commits exceed Facebook’s over the last few months.
While these numbers are exciting on their own, we’re also happy that we’ve been able to grow our portfolio to address the newest challenges that contemporary software engineers face. This year, we’ve made a concerted effort to help mobile developer communities, and have continued our relentless pursuit for infrastructure performance and stability. The themes of our open source platform now range from native mobile libraries to leading web and test frameworks, from server platforms to big data infrastructure, from prototype and IT tools to hardware and data center designs – and all points in between.
A push – and a Pop – into mobile
In January, we announced Facebook Paper, the first iOS app from the Facebook Creative Labs initiative.
Shortly afterwards, the Paper team started to open source many of the libraries that were built during the app’s development: KVOController for thread-safe key-value observing, Shimmer for unobtrusive loading indicators, and Tweaks, an easy way to fine-tune iOS apps during development.
Last, but definitely not least, we released the iOS and OS X animation library, Pop. It’s already become our second most popular project ever, and has enjoyed widespread usage, including in the iOS version of our very own Slingshot app. (The Android version of Slingshot features the similar project Rebound – which is also now used in many other well-known apps). We love contributing to and working with the community, but it’s also exciting to be able to use and improve these open source projects in our own production environments.
What’s next from the Paper team? At our f8 conference, you may have caught our introduction to AsyncDisplayKit, an asynchronous UI framework for iOS. Today we’re excited to announce we have open sourced the project and have opened up beta access. Please join the Paper Engineering Community to get involved and try it out.
Elsewhere in our mobile portfolio, we released Conceal, an Android library to encrypt data to SD cards, Chisel, a collection of LLDB debugging commands, and Bolts, a suite of low-level libraries that support various useful programming patterns, as well as the App Links initiative that launched in April.
React: JavaScript at scale
Just a few weeks ago, we celebrated the first anniversary of our flagship JavaScript library, React, which aims to support web developers with its reactive pattern, virtual DOM, and optional JSX syntax.
Never afraid to challenge a few long-held assumptions, the team behind React recently talked through its novel one-way data-flow philosophy (known as ‘Flux’) at our f8 conference. We’ve also worked hard to make the React development process easier than ever: we shipped the Dev Tools extension to expose the virtual DOM in the browser; bindings for Express, PHP/V8, and .NET; and editor support for Sublime
We also had an itch to scratch with JavaScript testing, and in May, we were excited to launch Jest, a framework built upon Jasmine to provide automatic CommonJS mocking, familiar assertion syntax, and asynchronous test runs.
Contributions to our web projects are now stronger than ever, and as the team’s regular round-ups show, we continue to see exciting React-related activity throughout the web community.
Move fast with stable infrastructure…
With its two-week release cycle iterating through an alphabet of hip-hop artists, we continue to iterate quickly on HHVM, our high-performance virtual machine. This is the runtime that powers Facebook, and it remains our biggest open source project in terms of followers, forks, and contributors.
HHVM is also more compatible than ever before. In January, we were passing just 6 major frameworks’ PHPUnit test suites. but we passed our goal of 20 in April, and are pleased to currently support 26 major frameworks with 100% pass rates.
In March, we announced the new programming language Hack, featuring gradual typing, many modern language features, and an easy migration path from PHP. Hack represents a strong long-term commitment for Facebook – we’ve converted our own entire PHP code base to use it – and we have been really pleased with the adoption already, as our recent community round-ups attest.
In February, we also released FBThrift, a recent update to Apache Thrift that includes a new, full-asynchronous, C++ server and numerous other improvements. Our stated intent is to upstream as many of these updates back to Apache as possible.
… big data …
Our work on open source data infrastructure continues apace.
In April, we announced the WebScaleSQL project, a collaboration between the database engineering teams at Facebook, Google, LinkedIn, and Twitter, and designed to help implement scale-oriented features and optimizations for MySQL. Again, we’re tracking upstream, and continue to share our updates and patches as they land.
Presto, our big data query engine, and RocksDB, our fast key-value store – both released late last year – have continued to grow. Both have very active communities, have had significant releases, and continue to gain new features: just this week, for example, we announced the new PlainTable file format for RocksDB. We also blogged recently about the work we’ve been putting into HBase and Giraph in order to run them successfully at Facebook scale.
And only this week, the Parse team made two strong contributions to the MongoDB community: Dvara, a connection pooling proxy, and Flashback, allowing the capture and replay of production workloads while in a development or test environment.
…and efficient developers
Infrastructure and data performance is clearly very important, but we’ve also worked on open sourcing a number of initiatives to improve developer performance this year. In January we wrote about how we are now using Mercurial for our source control, and we have now contributed over 1,000 patches to that project as well as a number of additional libraries to support large scale hg deployments.
Buck, our high performance build tool, is moving fast in more ways than one. It has benefitted from almost 1,000 commits since January, as well as new documentation. As a related initiative, we also forked Proguard, a Java optimizer, and improved its performance 2.5 times.
C and C++ developers can also enjoy new high-performance tooling for linting and pre-processing, in the form of Flint and Warp respectively – both of which were written in D. Most recently, we open sourced Haxl, a library for efficient concurrency in Haskell.
And at the other end of the spectrum, we are even using open source at the network layer and at the data-center scale. We just announced our new switching hardware and operating system – code named ‘Wedge’ and ‘FBOSS’ – both of which we will be sharing with the community in coming months. We also open-sourced our sustainability dashboard system, which allows easy visualization of the water and power consumption of data center facilities.
Out, about, and beyond
Many of our open source engineering teams are out and about in the coming months. We will be out in force at next month’s OSCON, and we hope to see you there for sessions on Hack & HHVM, the architecture behind React, the Instagram front-end, our open source Android projects, how we run our open source program, our engineering process, and Open Academy.
Today we’re also pre-announcing our @Scale 2014 engineering conference, which we’ll hold in San Francisco on Monday, September 15. On the heels of our popular Mobile @Scale and Data @Scale themed events, the conference will comprise a day-long series of deeply technical talks from a variety of leading companies, and focused on mobile, data, and web development at scale. Like this page and follow our engineering page and open source updates for forthcoming registration details.
We remain committed to a broad and well-balanced open source program, and maintaining a healthy and active relationship with our communities. Whether it’s on GitHub, on IRC, on mailing lists – or in real life at events – we’re excited to work with you. Together we believe we can use open source to push mobile, web, data and infrastructure technology forward, and to make all of these developer communities more open and connected.
So, um, who wants to commit that 10 millionth line of code?
