Learning-based frequency estimation algorithms
Name
learning_based_frequency_estimation_algorithms.pdf
Description
Accepted version
Size
756.94 KB
Format
Adobe PDF
Checksum (MD5)
08513000aef7890bddd81d52c53db151
Author(s)
Hsu, Chen-Yu
Indyk, Piotr
Katabi, Dina
Vakilian, Ali
Date Issued
May 2019
Journal
7th International Conference on Learning Representations, ICLR 2019
Publisher
ICLR
Citation
Hsu, Chen-Yu et al. “.” Paper presented at the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, Louisiana, May 6 - 9, 2019, ICLR: © 2019 The Author(s)
Version
Author's final manuscript
Abstract
Estimating the frequencies of elements in a data stream is a fundamental task in data analysis and machine learning. The problem is typically addressed using streaming algorithms which can process very large data using limited storage. Today's streaming algorithms, however, cannot exploit patterns in their input to improve performance. We propose a new class of algorithms that automatically learn relevant patterns in the input data and use them to improve its frequency estimates. The proposed algorithms combine the benefits of machine learning with the formal guarantees available through algorithm theory. We prove that our learning-based algorithms have lower estimation errors than their non-learning counterparts. We also evaluate our algorithms on two real-world datasets and demonstrate empirically their performance gains.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
