Advanced Search

An Efficient Fill Estimation Algorithm for Sparse Matrices and Tensors in Blocked Formats

Research and Teaching Output of the MIT Community

Show simple item record

dc.contributor.advisor Alan Edelman Ahrens, Peter en_US Schiefer, Nicholas en_US Xu, Helen en_US
dc.contributor.other Theory of Computation en 2017-06-12T15:45:16Z 2017-06-12T15:45:16Z 2017-06-09
dc.description.abstract Tensors, linear-algebraic extensions of matrices in arbitrary dimensions, have numerous applications in computer science and computational science. Many tensors are sparse, containing more than 90% zero entries. Efficient algorithms can leverage sparsity to do less work, but the irregular locations of the nonzero entries pose challenges to performance engineers. Many tensor operations such as tensor-vector multiplications can be sped up substantially by breaking the tensor into equally sized blocks (only storing blocks which contain nonzeros) and performing operations in each block using carefully tuned code. However, selecting the best block size is computationally challenging. Previously, Vuduc et al. defined the fill of a sparse tensor to be the number of stored entries in the blocked format divided by the number of nonzero entries, and showed that the fill can be used as an effective heuristic to choose a good block size. However, they gave no accuracy bounds for their method for estimating the fill, and it is vulnerable to adversarial examples. In this paper, we present a sampling-based method for finding a (1 + epsilon)-approximation to the fill of an order N tensor for all block sizes less than B, with probability at least 1 - delta, using O(N B^N log(B / delta) / epsilon^2) samples for each block size. We introduce an efficient routine to sample for all B^N block sizes at once in O(N B^N) time. We extend our concentration bounds to a more efficient bound based on sampling without replacement, using the recent Hoeffding-Serfling inequality. We then implement our algorithm and compare our scheme to that of Vuduc, as implemented in the Optimized Sparse Kernel Interface (OSKI) library. We find that our algorithm provides faster estimates of the fill at all accuracy levels, providing evidence that this is both a theoretical and practical improvement. Our code is available under the BSD 3-clause license at en_US
dc.format.extent 19 p. en_US
dc.relation.ispartofseries MIT-CSAIL-TR-2017-011
dc.subject Randomized Algorithm en_US
dc.subject Sampling Algorithm en_US
dc.subject Autotuning en_US
dc.subject Performance Engineering en_US
dc.subject Numerical Linear Algebra en_US
dc.title An Efficient Fill Estimation Algorithm for Sparse Matrices and Tensors in Blocked Formats en_US 2017-06-12T15:45:16Z

Files in this item

Name Size Format Description
MIT-CSAIL-TR-2017 ... 463.0Kb PDF

This item appears in the following Collection(s)

Show simple item record