Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. MIT Open Access Articles
  3. MIT Open Access Articles
  4. Lookup Tables: Fine-Grained Partitioning for Distributed Databases

Lookup Tables: Fine-Grained Partitioning for Distributed Databases

Thumbnail Image
Download
Name

lookup_tables.pdf

Description
Accepted version
Size

484.54 KB

Format

Adobe PDF

Checksum (MD5)

a9bac9711a2f8e5a92642f0cc365e3ff

sword-2019-06-18T13:07:02.original.xml (130 B)
Original SWORD entry document
Author(s)
Tatarowicz, Aubrey L.
•
Curino, Carlo
•
Jones, Evan P.C.
•
Madden, Sam
Date Issued
April 2012
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Tatarowicz, Aubrey L., Curino, Carlo, Jones, Evan P.C. and Madden, Sam. 2012. "Lookup Tables: Fine-Grained Partitioning for Distributed Databases."
Version
Author's final manuscript
Abstract
The standard way to get linear scaling in a distributed OLTP DBMS is to horizontally partition data across several nodes. Ideally, this partitioning will result in each query being executed at just one node, to avoid the overheads of distributed transactions and allow nodes to be added without increasing the amount of required coordination. For some applications, simple strategies, such as hashing on primary key, provide this property. Unfortunately, for many applications, including social networking and order-fulfillment, many-to-many relationships cause simple strategies to result in a large fraction of distributed queries. Instead, what is needed is a fine-grained partitioning, where related individual tuples (e.g., cliques of friends) are co-located together in the same partition. Maintaining such a fine-grained partitioning requires the database to store a large amount of metadata about which partition each tuple resides in. We call such metadata a lookup table, and present the design of a data distribution layer that efficiently stores these tables and maintains them in the presence of inserts, deletes, and updates. We show that such tables can provide scalability for several difficult to partition database workloads, including Wikipedia, Twitter, and TPC-E. Our implementation provides 40% to 300% better performance on these workloads than either simple range or hash partitioning and shows greater potential for further scale-out. © 2012 IEEE.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
http://creativecommons.org/licenses/by-nc-sa/4.0/
Persistent DSpace Link
https://hdl.handle.net/1721.1/137764
DOI of Published Version
https://doi.org/10.1109/icde.2012.26
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.