Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service
Name
3581784.3607034.pdf
Size
1.6 MB
Format
Adobe PDF
Checksum (MD5)
3bf44baedb084bf51be990b95f92e220
Author(s)
Li, Baolin
Samsi, Siddharth
Gadepally, Vijay
Tiwari, Devesh
Date Issued
November 12, 2023
Publisher
ACM|The International Conference for High Performance Computing, Networking, Storage and Analysis
Citation
Li, Baolin, Samsi, Siddharth, Gadepally, Vijay and Tiwari, Devesh. 2023. "Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service."
Version
Final published version
Abstract
This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML inference is critical to modern technology products, but it is also a significant contributor to carbon footprint. We introduce, Clover, a carbon-friendly ML inference service runtime system that balances performance, accuracy, and carbon emissions through mixed-quality models and GPU resource partitioning. Our experimental results demonstrate that Clover is effective in substantially reducing carbon emissions while maintaining high accuracy and meeting service level agreement (SLA) targets.
MIT Department
Lincoln Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3581784.3607034
