Distributed dynamic partially stateful dataflow
Name
1051460356-MIT.pdf
Description
Full printable version
Size
4.75 MB
Format
Adobe PDF
Checksum (MD5)
cbf8452dc4b50d8e1b04bab7a6dc33a1
Author(s)
Behrens, Jonathan (Jonathan Kyle)
Advisor(s)
M. Frans Kaashoek and Malte Schwarzkopf.
Date Issued
2018
Publisher
Massachusetts Institute of Technology
Abstract
This thesis present a distributed implementation of Noria, a new streaming dataflow system that simplifies the infrastructure of read-heavy web applications by unifying the database, caching layer, and parts of application logic in a single system. Noria's partially-stateful dataflow allows it to evict and reconstruct state on demand, and avoid prior dataflow systems' restriction to windowed state. Unlike existing dataflow systems, Noria adapts on-line to schema and query changes, and shares state and computation across related queries to eliminate duplicate effort. Noria's distributed design enables it to leverage the compute power of an entire cluster while providing high availability thanks to its fault tolerant design. On a single machine, Noria already outperforms MySQL by up to 7 x, but when running across a cluster of machines, it can scale to tens of millions of reads and millions of writes per second.
Description
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 43-49).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
Persistent DSpace Link
