LightSpeed: A Framework to Profile and Evaluate
Inference Accelerators at Scale
Name
williams-chrwill-meng-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
656.12 KB
Format
Adobe PDF
Checksum (MD5)
df91b39b9789eca28262d16407228df8
Author(s)
Williams, Christian
Advisor(s)
Ghobadi, Manya
Date Issued
June 2023
Publisher
Massachusetts Institute of Technology
Abstract
The massive growth of machine learning-based applications, and the end of Moore’s law, created a pressing need to build highly efficient computing platforms from the ground up. Consequently, researchers and practitioners have been developing highly innovative cutting-edge architectures to meet today’s exponentially increasing demands for machine learning services.
However, evaluating the performance gains of newly developed machine learning systems at scale is extremely challenging. Existing evaluation platforms are often specialized to a specific hardware target, such as GPUs, making them less amenable to novel designs. Moreover, evaluating the performance of a newly designed system at scale requires careful consideration of workload and traffic patterns.
To address the above challenges, I introduce LightSpeed, a framework to profile and evaluate inference accelerators at scale. LightSpeed is an event-based simulator that enables users to compare the performance of their system to best-in-class accelerators at scale. LightSpeed profiles the computation and communication requirements of real-world deep neural networks through accurate measurements on hardware. It then simulates the service time of inference requests under a variety of accelerators and scheduling algorithms.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
