Optimizing Data Layouts for Evolving Cloud Table Storage
Name
sudhir-ssudhir-phd-eecs-2025-thesis.pdf
Description
Thesis PDF
Size
17.6 MB
Format
Adobe PDF
Checksum (MD5)
c20d9b7a7d8392185e6dd27ee549589c
Author(s)
Sudhir, Sivaprasad
Advisor(s)
Cafarella, Michael J.
Madden, Samuel R.
Date Issued
September 2025
Publisher
Massachusetts Institute of Technology
Abstract
Modern data analytics platforms increasingly adopt disaggregated architectures, storing data in cost-effective cloud object stores. While this approach enables a clean separation of concerns, allowing each layer to be independently managed and scaled, it introduces significant performance bottlenecks due to expensive data movement. Effective data layouts, which organize data to minimize unnecessary data reads, are thus critical to achieving high query performance. However, existing techniques typically rely on manually specified layouts, collect limited metadata, or lack mechanisms to dynamically adapt to changing data and workloads.
This thesis investigates adaptive, metadata-rich, expressive data layouts for cloud table storage. First, we introduce Pando, a correlation-aware layout technique that leverages rich metadata on query predicates to significantly improve data skipping. Next, we propose CopyRight, a partial replication strategy that selectively replicates subsets of data and optimizes each replica differently, efficiently serving heterogeneous query patterns. Finally, we describe Self-Organizing Data Containers (SDCs), a practical table storage layer for the cloud that incrementally reorganizes complex data layouts based on changes in data and workload distributions.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
