Partitioning Techniques for Fine-grained Indexing
Name
Madden_Partitioning techniques.pdf
Size
847.21 KB
Format
Adobe PDF
Checksum (MD5)
def1c353226a1f4db4c6d84565eb76da
Author(s)
Wu, Eugene
Madden, Samuel R.
Date Issued
April 2011
Journal
Proceedings of the International Conference on Data Engineering, ICDE 2011
Publisher
International Conference on Data Engineering
Citation
Wu, Eugene and Samuel Madden. "Partitioning Techniques for Fine-grained Indexing." International Conference on Data Engineering, ICDE 2011, Hannover, April 11-16, 2011.
Version
Author's final manuscript
Abstract
Many data-intensive websites use databases that grow
much faster than the rate that users access the data. Such growing
datasets lead to ever-increasing space and performance overheads
for maintaining and accessing indexes. Furthermore, there is often
considerable skew with popular users and recent data accessed
much more frequently. These observations led us to design Shinobi,
a system which uses horizontal partitioning as a mechanism for
improving query performance to cluster the physical data, and increasing
insert performance by only indexing data that is frequently
accessed. We present database design algorithms that optimally
partition tables, drop indexes from partitions that are infrequently
queried, and maintain these partitions as workloads change. We
show a 60× performance improvement over traditionally indexed
tables using a real-world query workload derived from a traffic
monitoring application
Description
URL to paper listed on conference site
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
http://www.icde2011.org/node/94
