Tailors: Accelerating Sparse Tensor Algebra by Overbooking Buffer Capacity
Name
3613424.3623793.pdf
Size
1.31 MB
Format
Adobe PDF
Checksum (MD5)
50d7e8271fad597540b560b274b43ae1
Author(s)
Xue, Zi Yu
Wu, Yannan
Emer, Joel
Sze, Vivienne
Date Issued
October 28, 2023
Publisher
ACM|56th Annual IEEE/ACM International Symposium on Microarchitecture
Citation
Xue, Zi Yu, Wu, Yannan, Emer, Joel and Sze, Vivienne. 2023. "Tailors: Accelerating Sparse Tensor Algebra by Overbooking Buffer Capacity."
Version
Final published version
Abstract
Sparse tensor algebra is a challenging class of workloads to accelerate due to low arithmetic intensity and varying sparsity patterns. Prior sparse tensor algebra accelerators have explored tiling sparse data to increase exploitable data reuse and improve throughput, but typically allocate tile size in a given buffer for the worst-case data occupancy. This severely limits the utilization of available memory resources and reduces data reuse. Other accelerators employ complex tiling during preprocessing or at runtime to determine the exact tile size based on its occupancy.
This paper proposes a speculative tensor tiling approach, called overbooking, to improve buffer utilization by taking advantage of the distribution of nonzero elements in sparse tensors to construct larger tiles with greater data reuse. To ensure correctness, we propose a low-overhead hardware mechanism, Tailors, that can tolerate data overflow by design while ensuring reasonable data reuse. We demonstrate that Tailors can be easily integrated into the memory hierarchy of an existing sparse tensor algebra accelerator. To ensure high buffer utilization with minimal tiling overhead, we introduce a statistical approach, Swiftiles, to pick a tile size so that tiles usually fit within the buffer’s capacity, but can potentially overflow, i.e., it overbooks the buffers. Across a suite of 22 sparse tensor algebra workloads, we show that our proposed overbooking strategy introduces an average speedup of 52.7 × and 2.3 × and an average energy reduction of 22.5 × and 2.5 × over ExTensor without and with optimized tiling, respectively.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3613424.3623793
