Mixed-precision NN accelerator with neural-hardware architecture search
Author(s)Lin, Yujun,S. M.Massachusetts Institute of Technology.; Hafdi, Driss.
Mixed-precision neural network accelerator with neural-hardware architecture search
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
MetadataShow full item record
Neural architecture and hardware architecture co-design is an effective way to enable specialization and acceleration for deep neural networks (DNNs). The design space and its exploration methodology impact efficiency and productivity. However, both architecture designs are challenging. We first propose a mixed-precision accelerator, a highly parameterized architecture that can adapt to different bit widths for different quantized layers with significantly reduced overhead. It efficiently provides a vast design space for both neural and hardware architecture. However, it is difficult to exhaust such an enormous design space by rule-based heuristics. To tackle this problem, we propose a machine learning based design and optimization methodology of a neural network accelerator. It includes the evolution strategy based hardware architecture search and one-shot HyperNet based quantized neural architecture search. Evaluated on existing DNN benchmarks, our mixed-precision accelerator achieves 11.7x, 1.5x speedup and 10.5x, 1.9x energy savings over Eyeriss  and BitFusion  respectively under the same area, frequency, and process technology. Our machine learning based co-design can compose highly matched neural-hardware architectures and further rival the best human-designed architectures by additional 1.3x speedup and 1.5x energy savings under the same ImageNet accuracy with better sample efficiency.
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020Cataloged from the official PDF of thesis. "Part of the work in this thesis was done in collaboration with another student, Driss Hafdi. The credit for the design and implementation of accelerator architecture in this thesis was shared by both of us"--Page 5 Disclaimer.Includes bibliographical references (pages 61-65).
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Electrical Engineering and Computer Science.