What the research is:
New strategies to improve the performance of hardware platforms running Facebook’s microservices. SoftSKU is a novel mechanism that tailors an existing server processor to optimize it for a specific microservice without requiring any additional hardware. Accelerometer is an analytical model that predicts gains from these optimizations even before the custom hardware is installed.
The variety and complexity of microservices in warehouse-scale data centers has grown precipitously over the past few years to support a growing user base and evolving product portfolios. Prior works around optimizing data center microservices primarily involve developing customized hardware platforms to solve performance bottlenecks faced by modern microservices. However, large-scale internet operators have strong incentives to limit hardware platform diversity to preserve economies of scale. These new approaches enable efficient utilization of existing commodity hardware and help us make well-informed future hardware investments. As these tunable knobs become more prevalent in modern processors, we expect this work to become increasingly useful.
How it works:
SoftSKU is a strategy in which we exploit coarse-grained operating system and hardware configuration knobs (e.g., number of cores, prefetcher configuration, huge pages) to tune a given hardware platform to optimally run their assigned microservice. We developed a tool called μSKU that automatically designs and deploys soft SKUs via A/B testing. In our testing, soft SKUs outperformed stock and production server configurations by up to 7.2 percent, with no additional hardware requirement.
https://www.youtube.com/watch?v=m_SAiOQwu4w
Our diverse microservices have diverse system and CPU architectural requirements. Such diversity might suggest a strategy to specialize CPU architectures to suit each microservice’s distinct needs. Although developing specialized hardware acceleration for each microservice building block might be beneficial, it becomes prohibitive at hyperscale if these accelerators do not yield the expected gains due to performance bounds precipitated by offload-induced overheads. To this end, we developed Accelerometer, an analytical model that enables data center operators to estimate gains from hardware acceleration and make well-informed future hardware investments. We’ve validated Accelerometer using three retrospective case studies and demonstrate that its speedup estimates have an error rate of less than 3.7 percent.
Why it matters:
SoftSKU and Accelerometer have enabled us to customize available server processors and model future hardware in order to make well-informed hardware investments at hyperscale. We believe our research can help other large-scale organizations find similar gains.
The SoftSKU strategy establishes that it is possible to improve the performance efficiency of available general-purpose commodity hardware without additional hardware customization. When applied at hyperscale, SoftSKU can result in an up to 7.2 percent performance improvement.
Our Accelerometer analytical model is very powerful in terms of accurately understanding hardware specifications as well as how software interacts with the hardware to estimate realistic gains from hardware acceleration. Such analytical models are extremely valuable in helping data center operators make well-informed future hardware investments.
Read the full papers:
SoftSKU: Optimizing Server Architectures for Microservice Diversity @Scale
Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale
We would like to thank Akshitha Sriraman for her contribution in developing SoftSKU and Accelerometer and Thomas F. Wenisch for insights that helped improve the work.
