Mixed-precision NN accelerator with neural-hardware architecture search

Abstract

Neural architecture and hardware architecture co-design is an effective way to enable specialization and acceleration for deep neural networks (DNNs). The design space and its exploration methodology impact efficiency and productivity. However, both architecture designs are challenging. We first propose a mixed-precision accelerator, a highly parameterized architecture that can adapt to different bit widths for different quantized layers with significantly reduced overhead. It efficiently provides a vast design space for both neural and hardware architecture. However, it is difficult to exhaust such an enormous design space by rule-based heuristics. To tackle this problem, we propose a machine learning based design and optimization methodology of a neural network accelerator. It includes the evolution strategy based hardware architecture search and one-shot HyperNet based quantized neural architecture search. Evaluated on existing DNN benchmarks, our mixed-precision accelerator achieves 11.7x, 1.5x speedup and 10.5x, 1.9x energy savings over Eyeriss [3] and BitFusion [35] respectively under the same area, frequency, and process technology. Our machine learning based co-design can compose highly matched neural-hardware architectures and further rival the best human-designed architectures by additional 1.3x speedup and 1.5x energy savings under the same ImageNet accuracy with better sample efficiency.