Learning-based frequency estimation algorithms
Author(s)Hsu, Chen-Yu; Indyk, Piotr; Katabi, Dina; Vakilian, Ali
MetadataShow full item record
Estimating the frequencies of elements in a data stream is a fundamental task in data analysis and machine learning. The problem is typically addressed using streaming algorithms which can process very large data using limited storage. Today's streaming algorithms, however, cannot exploit patterns in their input to improve performance. We propose a new class of algorithms that automatically learn relevant patterns in the input data and use them to improve its frequency estimates. The proposed algorithms combine the benefits of machine learning with the formal guarantees available through algorithm theory. We prove that our learning-based algorithms have lower estimation errors than their non-learning counterparts. We also evaluate our algorithms on two real-world datasets and demonstrate empirically their performance gains.
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
7th International Conference on Learning Representations, ICLR 2019
Hsu, Chen-Yu et al. “.” Paper presented at the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, Louisiana, May 6 - 9, 2019, ICLR: © 2019 The Author(s)
Author's final manuscript