Workload-Driven Network Optimization for Distributed Machine Learning
Name
wang-weiyangw-phd-eecs-2026-thesis.pdf
Size
10.02 MB
Format
Adobe PDF
Checksum (MD5)
d9a89ecbc6c4a26d2b61817d2bf5f365
Author(s)
Wang, Weiyang
Advisor(s)
Ghobadi, Manya
Date Issued
May 2026
Publisher
Massachusetts Institute of Technology
Abstract
Modern machine learning workloads require networks that connect tens to hundreds of thousands of GPUs. At this scale, the network determines two of the most critical properties of a training cluster: how efficiently GPUs communicate, and how the cluster tolerates the failures that become routine at scale. Existing GPU clusters rely on network designs that provide any-to-any connectivity while remaining agnostic to the data they carry. These properties are inherited from legacy CPU datacenters and limit scalability while hindering GPU utilization. This dissertation presents workload-driven networking: a design approach that specializes GPUcluster networks to the structure of ML workloads. The central observation is that ML traffic is not generic datacenter traffic. Its endpoints are sparse, because communication concentrates among a small set of GPU pairs; its patterns are predictable and repetitive, because they are determined by the parallelization strategy before training begins; and its contents are meaningful, because gradients and activations encode training state. Conventional any-to-any, content-oblivious fabrics hide these properties behind a uniform network abstraction, limiting both efficiency and reliability at scale. This dissertation explores an alternative in which network architecture, topology, and datapath function are co-designed with the training workload. Rail-only networks remove the spine layer from LLM training clusters, reducing network cost by 38%–77% and power by 37%–75% while preserving training performance. TopoOpt uses reconfigurable optics to tailor connectivity to each training job, achieving up to 3.4× faster iterations than a Fat-tree with the same server degree. Checkmate uses reduced gradients already crossing the network to maintain a shadow copy of model state, enabling per-iteration checkpointing with zero GPU overhead and 5×–34.5× more checkpoints than state-of-the-art checkpointing systems. Together, these systems demonstrate that exploiting workload structure is a productive approach to designing ML networks. As ML workloads grow more demanding and specialized, ad-hoc, workload-agnostic fabrics will become increasingly untenable, and we expect workload-driven co-design to become the primary methodology for ML networking and systems. Future research directions include extending the approach to inference, whose traffic patterns and latency requirements differ sharply from training, and automating the co-design itself by building interfaces through which workloads expose their structure and networks specialize to it without per-system human effort.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
Persistent DSpace Link
