Mitigating Compute Congestion for Low Latency Datacenter RPCs
Name
cho-inhocho-phd-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
6.44 MB
Format
Adobe PDF
Checksum (MD5)
4b5a4597b7f15d353d7ad5dae77a9e7b
Author(s)
Cho, Inho
Advisor(s)
Belay, Adam M.
Alizadeh, Mohammad
Date Issued
September 2023
Publisher
Massachusetts Institute of Technology
Abstract
Latency-sensitive applications in recent datacenter workloads, such as interactive machine learning inference, high-frequency algorithm trading, cloud gaming, and interactive AR/VR applications impose stringent latency requirements. These applications heavily rely on low-latency RPCs as an essential building block, often executed in mere microseconds through parallel computations and in-memory operations. Given the high fan-out RPC traffic patterns typical of these applications, it’s imperative to minimize tail latency to maintain end-to-end latency within its service level objectives (SLO).
With the innovations in datacenter networks and the end of Dennard scaling, congestion is now moving from networks to compute resources. This thesis introduces two systems, Breakwater and LDB, designed to mitigate and diagnose compute congestion, each targeting different sources of tail latency. Breakwater aims to alleviate CPU congestion and lock contention during intermittent server overload, while LDB furnishes developers with a tool to diagnose the functions causing high tail latency with low overhead.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
