Defio: Instance-Optimized Fusion of AWS Database Services
Name
fanggohans-dfangs-meng-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
745.52 KB
Format
Adobe PDF
Checksum (MD5)
46b7fafa98c985a0dbe2e8ce933de37f
Author(s)
Fanggohans, Dean
Advisor(s)
Kraska,Tim
Date Issued
June 2023
Publisher
Massachusetts Institute of Technology
Abstract
Building large-scale data infrastructures is hard: There are often more than a single type of workloads and business requirements, but unfortunately, “one size does not fit all”. Modern database systems tend to specialize towards a specific type of workload, and thus organizations are left to integrate these differently-specialized database systems in order to achieve sufficient performance for all of their use cases and workloads.
This kind of hybrid architecture—also known as Data Mesh architecture—often leads to the increasing complexity of maintaining and utilizing database services, both for the data engineers and the end users. However, we believe that some of this complexity can be abstracted away from the end users, in particular with respect to query routing, i.e. determining where to execute each individual SQL query among the multiple database engines.
To overcome this challenge, we propose Defio, a unified interface to multiple specialized database engines that can intelligently handle myriads of workloads without having the end users think about the underlying execution of each query. Specifically, this thesis focuses on the design and implementation of an instance-optimized query router, which ultimately enables Defio to take advantage of the performance benefits of each specialized database in a Data Mesh architecture—resulting in what we call a fusion of database services.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
