Next Generation Operating Systems for the Datacenter

Abstract

Modern datacenters face a fundamental challenge: handling demanding real-time and dataintensive workloads that require both microsecond-scale low latency and high throughput, while simultaneously achieving high resource utilization and efficient multi-tenancy. Traditional operating systems, designed for an era of slower hardware, introduce significant overheads to microsecond-scale I/O that prevent applications from exploiting the full performance of the underlying hardware. Furthermore, their millisecond-scale resource management is ill-equipped to handle the microsecond-level burstiness of modern workloads, leading to costly overprovisioning and idle resources. Recognizing the performance limitations imposed by traditional OSes, a common workaround has emerged: letting applications communicate directly with hardware, bypassing the OS entirely. While this approach offers performance gains by removing the OS from the critical path, existing kernel-bypass solutions require dedicated resources, extensive application rewrites, and provide weak isolation, making them unsuitable for general-purpose, shared environments. This thesis presents a new datacenter operating system, composed of three integrated systems: Shenango, Caladan, and Junction. Together, they preserve the high-performance, low-overhead I/O benefits of kernel bypass, while providing efficient resource management, strong isolation for multi-tenant workloads, and compatibility with unmodified software. First, Shenango enables applications to bypass traditional OS-mediated I/O without dedicating CPU cores solely to polling. Next, Caladan ensures that idle resources can be used productively by other applications by actively managing competition for microarchitectural resources, thereby preserving each application’s high I/O performance and responsiveness. Finally, Junction overcomes several common limitations of kernel-bypass solutions, bringing these benefits to all applications by preserving compatibility with existing software and reducing memory and polling overheads. Collectively, these systems provide the advantages of direct hardware access without sacrificing the flexibility or efficiency of a general-purpose operating system. This work demonstrates that high I/O performance, efficient resource utilization, and broad application compatibility can indeed coexist, paving the way for a new generation of datacenter operating systems.