A little over a year ago, Facebook released Thrift as open source software. (See the original announcement.) Thrift is a lightweight software framework for enabling communication between programs written in different programming languages, running on different computers, or both. We decided to release it because we thought it could help other groups solve some of the same technical problems we have faced, and because we hoped that developers who found it useful would contribute improvements back to the project. A lot has happened in the year since we released Thrift. First and foremost, Thrift has gained a lot of cool features:
- Support for C#, Perl, Objective C, Erlang, Smalltalk, OCaml, and Haskell.
- More idiomatic style in Java and Ruby.
- Two new protocols: one for dense encoding and one using JSON syntax.
- Significant speed boost in Python using a C module and PHP using an extension.
As we had hoped, a lot of these features were developed not inside Facebook, but by other companies and individuals using Thrift and contributing their changes back to the project. Thanks so much to our biggest corporate contributors: Powerset, imeem, Evernote, and Amie Street, as well as our many individual contributors. I’m even more excited about some of the stuff being worked on now.
Here at Facebook, we’re working on a fully asynchronous client and server for C++. This server uses event-driven I/O like the current TNonblockingServer, but its interface to the application code is all based on asynchronous callbacks. This will allow us to write servers that can service thousands of simultaneous requests (each of which requires making calls to other Thrift or Memcache servers) with only a few threads.
Powerset’s Chad Walters and I are working on templatizing the C++ API. This will be an almost entirely backward-compatible change that will preserve all the flexibility of the Thrift API, but it will allow performance-conscious developers to add type-annotations that will improve serialization and deserialization speed by about 5x and 3x respectively.
Thrift’s Ruby mapping, which never got much attention here at Facebook, has had a surge of popularity amongst our external contributors. It’s getting some much-needed attention from Powerset’s Kevin Clark and our newest corporate contributor: RapLeaf. They’ve already got an accelerator extension in testing (which works like the existing Python and PHP accelerators) and are working on some serious style overhauls. At least, that’s what they tell me. I don’t know Ruby, so I mostly leave them alone. 🙂
Ross McFarland has been working on a C mapping for Thrift using glib. A C mapping has been one of our oft-requested features, so it’s great to see this finally taking shape.
There are a host of other features that are “on the back burner” for now, but which I expect to be incorporated eventually. These include patches that we received for an asynchronous Perl client, an SSL transport for C++ (based on GNU TLS), and a more robust file-based transport.
I think the most exciting thing going on with Thrift right now is our acceptance into the Apache Incubator. For those who are not familiar with it, the Apache Incubator is a program for open source projects to integrate themselves into the Apache Software Foundation in preparation for becoming a real Apache project (or sub-project). This is a great opportunity for Thrift. Becoming an Apache project will get us a lot of attention and brand-recognition. We are hoping it will pave the way for getting Thrift integrated into a lot of other open source projects, especially those that are already Apache projects themselves. Apache is quickly becoming the de facto home for large-scale open source server software, so we think this could be the perfect long-term home for Thrift. If you are interested in Thrift, the best documentation is still the original whitepaper. You can also check out Thrift’s new homepage. Most of the interesting updates are on the Thrift mailing lists (subscription info on the homepage). Thrift’s Subversion repository has just moved into the Apache Incubator repository. Information on accessing it is available on the Thrift homepage. A lot of experimental development is published in the unofficial Git repository.
