Towards Resource-Efficient Compound AI Systems
Name
3713082.3730377.pdf
Size
951.89 KB
Format
Adobe PDF
Checksum (MD5)
2e25b11af3101686eaba89be85d480dd
Author(s)
Chaudhry, Gohar Irfan
Choukse, Esha
Goiri, ??igo
Fonseca, Rodrigo
Belay, Adam
Bianchini, Ricardo
Date Issued
June 6, 2025
Publisher
ACM|Workshop on Hot Topics in Operating Systems
Citation
Gohar Irfan Chaudhry, Esha Choukse, Íñigo Goiri, Rodrigo Fonseca, Adam Belay, and Ricardo Bianchini. 2025. Towards Resource-Efficient Compound AI Systems. In Proceedings of the 2025 Workshop on Hot Topics in Operating Systems (HotOS '25). Association for Computing Machinery, New York, NY, USA, 218–224.
Version
Final published version
Abstract
Compound AI Systems, integrating multiple interacting components like models, retrievers, and external tools, have emerged as essential for addressing complex AI tasks. However, current implementations suffer from inefficient resource utilization due to tight coupling between application logic and execution details, a disconnect between orchestration and resource management layers, and the perceived exclusiveness between efficiency and quality.
We propose a vision for resource-efficient Compound AI Systems through a declarative workflow programming model and an adaptive runtime system for dynamic scheduling and resource-aware decision-making. Decoupling application logic from low-level details exposes levers for the runtime to flexibly configure the execution environment and resources, without compromising on quality. Enabling collaboration between the workflow orchestration and cluster manager enables higher efficiency through better scheduling and resource management.
We are building a prototype system, called Murakkab, to realize this vision. Our preliminary evaluation demonstrates speedups up to ~ 3.4× in workflow completion times while delivering ~ 4.5× higher energy efficiency, showing promise in optimizing resources and advancing AI system design.
Description
HOTOS 25, May 14–16, 2025, Banff, AB, Canada
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3713082.3730377
