MacroBase: Prioritizing Attention in Fast Data
Name
macrobase-sigmod2017(1).pdf
Description
Accepted version
Size
1.06 MB
Format
Adobe PDF
Checksum (MD5)
217fb477410b6688634955276384adbc
Author(s)
Abuzaid, Firas
Bailis, Peter
Ding, Jialin
Gan, Edward
Madden, Samuel
Narayanan, Deepak
Rong, Kexin
Suri, Sahaana
Date Issued
2018
Journal
ACM Transactions on Database Systems
Publisher
Association for Computing Machinery (ACM)
Version
Author's final manuscript
Abstract
© 2018 Association for Computing Machinery. As data volumes continue to rise, manual inspection is becoming increasingly untenable. In response, we present MacroBase, a data analytics engine that prioritizes end-user attention in high-volume fast data streams. MacroBase enables eficient, accurate, and modular analyses that highlight and aggregate important and unusual behavior, acting as a search engine for fast data. MacroBase is able to deliver order-of-magnitude speedups over alternatives by optimizing the combination of explanation (i.e., feature selection) and classification tasks and by leveraging a new reservoir sampler and heavy-hitters sketch specialized for fast data streams. As a result, MacroBase delivers accurate results at speeds of up to 2M events per second per query on a single core. The system has delivered meaningful results in production, including at a telematics company monitoring hundreds of thousands of vehicles.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3276463
