Practical Prediction Models for Query Optimization and
Scheduling in Modern Database Systems
Name
wu-ziniuw-phd-eecs-2026-thesis.pdf
Size
8.06 MB
Format
Adobe PDF
Checksum (MD5)
6ca8d67db0e6b5cc752d2a385550b3d0
Author(s)
Wu, Ziniu
Advisor(s)
Kraska, Tim
Madden, Samuel
Date Issued
May 2026
Publisher
Massachusetts Institute of Technology
Abstract
Database management systems (DBMSs) are the backbone of modern enterprises. Query optimizers and schedulers are core components of a DBMS that make critical execution decisions and significantly impact overall system performance. The fundamental challenge in designing an effective optimizer or scheduler is to accurately and efficiently estimate the performance impact of different decisions to choose the optimal one. Query optimizers rely on cardinality estimates to evaluate candidate execution plans, while query schedulers depend on predictions of query execution time to determine execution order and resource allocation. Over the past decade, the research community has proposed a wide range of machine learning approaches to improve the accuracy of these predictions. However, due to high training and maintenance costs, high inference latency, and limited robustness to workload and data drift, most existing ML-based predictors are not suitable for deployment on the critical path of query execution. In this thesis, we explore practical prediction models for query optimization and scheduling through system-model co-design principles. Rather than optimizing solely for prediction accuracy, we jointly design predictors with the downstream planning and scheduling mechanisms to maximize end-to-end query performance. We further focus predictive effort on components where accurate estimation has the greatest impact, thereby achieving a balance among effectiveness, efficiency, and robustness. Following these principles, we develop three practical methods: FactorJoin for query optimization, and Stage and IconqSched for query scheduling. These systems achieve comparable or better end-to-end query performance than prior state-of-the-art ML-based approaches, while maintaining a lightweight model with fast and robust predictions for practical deployment. Our predictors are integrated and deployed on the critical path of commercial DBMSs, yielding significant improvements in query performance and demonstrating that carefully designed learned models can be both practical and effective inside production database systems.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
