Today, we’re proud to announce a first, experimental release of h2tp, or the “HH (Hack) Transpiler,” a tool which allows projects that have converted from PHP to Hack to still make releases that target the PHP language.
Since the launch of Hack, many community members have asked us how to manage forward compatibility. Hack is backwards-compatible with PHP – if you’re running PHP on HHVM, Hack code will seamlessly integrate with it. But the inverse is not true. Once a project has converted to Hack, there is no way to run that Hack code with the PHP5 engine. Anyone who wants to use that project needs to be able to run HHVM to run the Hack code – and leave in the dust any existing users who aren’t able to make the leap for whatever reason. This is not something most established projects are willing to do for all but the most major of major releases, and for good reason.
Because the Hack language is so close to PHP, it’s reasonable to ask if you can just mechanically convert from Hack back into PHP. Erasing type annotations is easy, but Hack is much more than just its type system. It contains many features such as collections and short lambda expressions that cannot simply be erased in order to get back valid PHP. Furthermore, any such tool would be an inferior experience to using Hack directly on HHVM, and the team didn’t want converted code to be the canonical way Hack was used. We hoped that we would be able to find some better solution to the problem – one that would not run into the aforementioned issues.
So, we’ve spent the last couple of months building h2tp. It is a combination of a command line tool that transpiles your entire Hack project into PHP, and an additional library of PHP functions and classes meant to be used alongside the transpiled output. It is simple to use, and is invoked like so:
h2tp <path_to_input_dir> <path_to_output_dir>Along with this transformed output, we provide the “hacklib” library which should be included in the transformed project. The suggested workflow is to integrate h2tp with your build steps to ensure that with every version of your Hack project, you provide the corresponding php version. This should allow you to continue working with the Hack version of your project, using the latest and greatest features, while still ensuring that your users who are still running on a PHP5 engine are able to run your project.
What does h2tp do?
The simplest way to understand what h2tp does is to look at a simple example of Hack code that’s transformed by this tool. Here’s an example, where it only has to erase type annotation:
// Original Hack code
class Alice extends Person {
private int $height = 62;
private ?Vector<Person> $siblings;
public function transform(
Transformable $item
) : this {
if ($item instanceof Potion) {
$this->height = 10;
}
// ...
return $this;
}
}
// Transformed code from h2tp
class Alice extends Person {
private $height = 62;
private $siblings;
public function transform(
$item
) {
if ($item instanceof Potion) {
$this->height = 10;
}
// ...
return $this;
}
}
Here is another more interesting example, where h2tp must replace short lambdas with anonymous functions and list out all the variables that have been captured in scope:
// Original Hack code
function wonderland($drinkMe) {
$eatMe = getFood();
$transform =
($alice) ==>
$alice
->transform($drinkMe)
->transform($eatMe);
}
// Transformed code from h2tp
function wonderland($drinkMe) {
$eatMe = getFood();
$transform =
function ($alice)
use ($eatMe, $drinkMe) {
return $alice
->transform($drinkMe)
->transform($eatMe);
};
}
Challenges in implementation
Our overarching design goals with this tool were to make sure that transformed code would always behave exactly as untransformed code would, and that we provide an experimental version of the tool as soon as possible to the community at large. One of the simplifications that we chose to make for this first version of the transpiler was to only make use of local information in scope while transforming code in order to avoid whole program analysis or extensive type checker support. This has still made it possible for us to support most of Hack’s interesting features, but has on occasion provided some challenges. Supporting the collections library was one of the more tricky aspects of this process – we could not simply convert Hack collections to arrays, since Hack collections actually have reference semantics rather than value semantics. Hence, the hacklib provided along with this tool contains implementations of the entire collections library.
But transforming Hack code using collections is not merely about using equivalent constructs from the hacklib library. For example, on HHVM, empty Hack collections universally convert to false when used in a boolean context, something a standard PHP object can never do. This means that any arbitrary expression or statement that involves a boolean comparison, or any cast, might invoke this special rule if it involves a collection. So unless the expression being treated as boolean is trivially not a collection, we have to treat it as if it might potentially contain a collection. Hence, we have to be very cautious when making transformations.
// Original Hack code
function chopHead($subject) {
if (!$subject->isQueen()) {
$subject->removeHead();
}
}
// Transformed code from h2tp
function chopHead($subject) {
// isQueen could
// return a collection!
if (!\hacklib_cast_as_boolean(
$subject->isQueen()
)) {
$subject->removeHead();
}
}
In a few rare cases we actually have to choose not to support a feature. For instance, we do not support Instance variables using literal syntax in the class declaration.
class Blah { private static Vector<string> $v1 = Vector {}; // This is supported private Vector<string> $v2 = Vector {}; // This is not supported }
The reason we cannot support this construct is that our transformations require us to replace such literal syntax with an equivalent call to a constructor. Static collections can be initialized after the class declaration, but instance declarations would have to lie in the constructor. But classes may not always contain constructors and may inherit them. Without whole program analysis it’s impossible for us to know about the constructor that is being inherited. Is it final? Is it private? What are the parameters? Of course, in cases like this, h2tp does not just silently transpile to code that will fail, but will rather stop and yell.
Conversion Error: File "path_to_file.php", line 4, characters 26-27: Collection initializers in instance variables are currently not supported.
The documentation provided with this tool highlights what we support and what we don’t. You will notice that the list of things we do not support is very small, and in the long run we do hope to support all these constructs.
Finally, we want to emphasize that while h2tp is a great way to maintain compatibility with existing installed users when doing releases, it’s not the best way to work with Hack on a daily basis. Its output is machine-generated; while it’s close enough to the original source that investigating bug reports filed by people using it won’t be a problem, it’s also not something intended to be manually edited, or for edits to be preserved in any way.
The use case that h2tp excels at is as a final build step for a project to make a backwards-compatible release. The ideal is for a project to convert to Hack, have its source of truth be in Hack, have its contributors work in Hack day-to-day, etc., and then only run h2tp as a final build step to create an alternate version of a release. When using h2tp like this, an existing project can get all the benefits of converting to Hack with none of the drawbacks of leaving existing users behind. Right now, h2tp is still experimental – try it out and give us feedback!
