Characterizing a Memory Allocator at Warehouse Scale
Name
3620666.3651350.pdf
Size
2.06 MB
Format
Adobe PDF
Checksum (MD5)
86aca8ace7c7b0d4cb1347a90cf683ba
Author(s)
Zhou, Zhuangzhuang
Gogte, Vaibhav
Vaish, Nilay
Kennelly, Chris
Xia, Patrick
Kanev, Svilen
Moseley, Tipp
Delimitrou, Christina
Ranganathan, Parthasarathy
Date Issued
April 27, 2024
Publisher
ACM
Citation
Zhou, Zhuangzhuang, Gogte, Vaibhav, Vaish, Nilay, Kennelly, Chris, Xia, Patrick et al. 2024. "Characterizing a Memory Allocator at Warehouse Scale."
Version
Final published version
Abstract
Memory allocation constitutes a substantial component of warehouse-scale computation. Optimizing the memory allocator not only reduces the datacenter tax, but also improves application performance, leading to significant cost savings.
We present the first comprehensive characterization study of TCMalloc, a memory allocator used by warehouse-scale applications in Google's production fleet. Our characterization reveals a profound diversity in the memory allocation patterns, allocated object sizes and lifetimes, for large-scale datacenter workloads, as well as in their performance on heterogeneous hardware platforms. Based on these insights, we optimize TCMalloc for warehouse-scale environments. Specifically, we propose optimizations for each level of its cache hierarchy that include usage-based dynamic sizing of allocator caches, leveraging hardware topology to mitigate inter-core communication overhead, and improving allocation packing algorithms based on statistical data. We evaluate these design choices using benchmarks and fleet-wide A/B experiments in our production fleet, resulting in a 1.4% improvement in throughput and a 3.4% reduction in RAM usage for the entire fleet. For the applications with the highest memory allocation usage, we observe up to 8.1% and 6.3% improvement in throughput and memory usage respectively. At our scale, even a single percent CPU or memory improvement translates to significant savings in server costs.
Description
ASPLOS '24: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems April 27-May 1, 2024 La Jolla, CA, USA
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3620666.3651350
