S-Store: streaming meets transaction processing
Author(s)Meehan, John; Pavlo, Andrew; Tufte, Kristin; Zdonik, Stan; Aslantas, Cansu; Cetintemel, Ugur; Du, Jiang; Kraska, Tim; Maier, David; Tatbul Bitim, Emine Nesime; Madden, Samuel R; Stonebraker, Michael; Wang, Hao; ... Show more Show less
MetadataShow full item record
Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. Heretofore, these two modes of operation existed in separate, stove-piped systems. In this work, we attempt to fuse the two computational paradigms in a single system called S-Store. In this way, S-Store can simultaneously accommodate OLTP and streaming applications. We present a simple transaction model for streams that integrates seamlessly with a traditional OLTP system, and provides both ACID and stream-oriented guarantees. We chose to build S-Store as an extension of H-Store - an open-source, in-memory, distributed OLTP database system. By implementing S-Store in this way, we can make use of the transaction processing facilities that H-Store already provides, and we can concentrate on the additional features that are needed to support streaming. Similar implementations could be done using other main-memory OLTP platforms. We show that we can actually achieve higher throughput for streaming workloads in S-Store than an equivalent deployment in H-Store alone. We also show how this can be achieved within H-Store with the addition of a modest amount of new functionality. Furthermore, we compare S-Store to two state-of-the-art streaming systems, Esper and Apache Storm, and show how S-Store can sometimes exceed their performance while at the same time providing stronger correctness guarantees.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Proceedings of the VLDB Endowment
Association for Computing Machinery
Meehan, John, et al. “S-Store: Streaming Meets Transaction Processing.” Proceedings of the VLDB Endowment, vol. 8, no. 13, Sept. 2015, pp. 2134–45.
Final published version