HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity
Name
3613424.3623786.pdf
Size
3.26 MB
Format
Adobe PDF
Checksum (MD5)
22d6fedaa7d4dfc56492065d46fa07b4
Author(s)
Wu, Yannan
Tsai, Po-An
Muralidharan, Saurav
Parashar, Angshuman
Sze, Vivienne
Emer, Joel
Date Issued
October 28, 2023
Publisher
ACM|56th Annual IEEE/ACM International Symposium on Microarchitecture
Citation
Wu, Yannan, Tsai, Po-An, Muralidharan, Saurav, Parashar, Angshuman, Sze, Vivienne et al. 2023. "HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity."
Version
Final published version
Abstract
Due to complex interactions among various deep neural network (DNN) optimization techniques, modern DNNs can have weights and activations that are dense or sparse with diverse sparsity degrees. To offer a good trade-off between accuracy and hardware performance, an ideal DNN accelerator should have high flexibility to efficiently translate DNN sparsity into reductions in energy and/or latency without incurring significant complexity overhead.
This paper introduces hierarchical structured sparsity (HSS), with the key insight that we can systematically represent diverse sparsity degrees by having them hierarchically composed from multiple simple sparsity patterns. As a result, HSS simplifies the underlying hardware since it only needs to support simple sparsity patterns; this significantly reduces the sparsity acceleration overhead, which improves efficiency. Motivated by such opportunities, we propose a simultaneously efficient and flexible accelerator, named HighLight, to accelerate DNNs that have diverse sparsity degrees (including dense). Due to the flexibility of HSS, different HSS patterns can be introduced to DNNs to meet different applications’ accuracy requirements. Compared to existing works, HighLight achieves a geomean of up to 6.4 × better energy-delay product (EDP) across workloads with diverse sparsity degrees, and always sits on the EDP-accuracy Pareto frontier for representative DNNs.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3613424.3623786
