Cackle: Analytical Workload Cost and Performance Stability With Elastic Pools
Author(s)
Perron, Matthew
Castro Fernandez, Raul
Dewitt, David
Cafarella, Michael
Madden, Samuel
Date Issued
December 12, 2023
Journal
Proceedings of the ACM on Management of Data
Publisher
ACM
Citation
Perron, Matthew, Castro Fernandez, Raul, Dewitt, David, Cafarella, Michael and Madden, Samuel. 2023. "Cackle: Analytical Workload Cost and Performance Stability With Elastic Pools." Proceedings of the ACM on Management of Data, 1 (4 (SIGMOD)).
Version
Final published version
Abstract
Analytical query workloads are prone to rapid fluctuations in resource demands. These rapid, hard to predict resource demand changes make provisioning a challenge. Users must either over provision at excessive cost or suffer poor query latency when demand spikes. Prior work shows the viability of using cloud functions to match the supply of compute to the workload demand without provisioning resources ahead of time. For low query volumes, this approach is less costly at reasonable performance compared to provisioned systems, but as query volumes increase the cost overhead of cloud functions outweighs the benefit gained by rapid elasticity. In this work, we propose a novel strategy combining rapidly scalable but expensive resources with slow to start but inexpensive virtual machines to gain the benefit of elasticity without losing out on the cost savings of provisioned resources. We demonstrate a technique that minimizes cost over a wide range of workloads, environmental conditions, and compute costs while providing stable query performance. We implement these ideas in Cackle and demonstrate that it achieves similar performance and cost per query across a wide range of workloads, avoiding the cost and performance cliffs of alternative approaches.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3626720
