Advanced Search
DSpace@MIT

An Efficient Fill Estimation Algorithm for Sparse Matrices and Tensors in Blocked Formats

Research and Teaching Output of the MIT Community

Show simple item record

dc.contributor.advisor Alan Edelman
dc.contributor.author Ahrens, Peter en_US
dc.contributor.author Schiefer, Nicholas en_US
dc.contributor.author Xu, Helen en_US
dc.contributor.other Theory of Computation en
dc.date.accessioned 2017-06-12T15:45:16Z
dc.date.available 2017-06-12T15:45:16Z
dc.date.issued 2017-06-09
dc.identifier.uri http://hdl.handle.net/1721.1/109792
dc.description.abstract Tensors, linear-algebraic extensions of matrices in arbitrary dimensions, have numerous applications in computer science and computational science. Many tensors are sparse, containing more than 90% zero entries. Efficient algorithms can leverage sparsity to do less work, but the irregular locations of the nonzero entries pose challenges to performance engineers. Many tensor operations such as tensor-vector multiplications can be sped up substantially by breaking the tensor into equally sized blocks (only storing blocks which contain nonzeros) and performing operations in each block using carefully tuned code. However, selecting the best block size is computationally challenging. Previously, Vuduc et al. defined the fill of a sparse tensor to be the number of stored entries in the blocked format divided by the number of nonzero entries, and showed that the fill can be used as an effective heuristic to choose a good block size. However, they gave no accuracy bounds for their method for estimating the fill, and it is vulnerable to adversarial examples. In this paper, we present a sampling-based method for finding a (1 + epsilon)-approximation to the fill of an order N tensor for all block sizes less than B, with probability at least 1 - delta, using O(N B^N log(B / delta) / epsilon^2) samples for each block size. We introduce an efficient routine to sample for all B^N block sizes at once in O(N B^N) time. We extend our concentration bounds to a more efficient bound based on sampling without replacement, using the recent Hoeffding-Serfling inequality. We then implement our algorithm and compare our scheme to that of Vuduc, as implemented in the Optimized Sparse Kernel Interface (OSKI) library. We find that our algorithm provides faster estimates of the fill at all accuracy levels, providing evidence that this is both a theoretical and practical improvement. Our code is available under the BSD 3-clause license at https://github.com/peterahrens/FillEstimation. en_US
dc.format.extent 19 p. en_US
dc.relation.ispartofseries MIT-CSAIL-TR-2017-011
dc.subject Randomized Algorithm en_US
dc.subject Sampling Algorithm en_US
dc.subject Autotuning en_US
dc.subject Performance Engineering en_US
dc.subject Numerical Linear Algebra en_US
dc.title An Efficient Fill Estimation Algorithm for Sparse Matrices and Tensors in Blocked Formats en_US
dc.date.updated 2017-06-12T15:45:16Z


Files in this item

Name Size Format Description
MIT-CSAIL-TR-2017 ... 463.0Kb PDF

This item appears in the following Collection(s)

Show simple item record

MIT-Mirage