LoopTree: Enabling Systematic and Flexible Exploration of Fused-layer Dataflow Accelerators

Abstract

Deep neural network (DNN) accelerators exploit data reuse to reduce memory traffic. Typically, DNN accelerators exploit data reuse within layers. However, there is also reuse between layers. To exploit this inter-layer reuse opportunity, fused-layer dataflow accelerators tile and buffer intermediate data between layers on-chip to benefit from inter-layer reuse while minimizing buffer size. To further minimize buffer space requirement, some fused-layer dataflows also propose not buffering part of the tile at the cost of recomputation of the unbuffered data.

The design space of fused-layer dataflow accelerators is large, but prior work only considers a subset of the design space. Prior works are limited in a number of ways: (1) tiling only in certain dimensions, leaving some designs unexplored; (2) limited choices of reuse/recompute which are applied uniformly to all layers, leading to increased recomputation; (3) not exploring the interaction of tiling and reuse/recompute choices; and (4) applying the same design choices for all layers in the DNN despite diverse layer shapes, which call for different choices.

To address these limitations, we propose (1) a more extensive design space, (2) a taxonomy that introduces structure into the design space, and (3) a fast, flexible, analytical model, called LoopTree, to evaluate the latency, energy consumption, buffer space requirements, and bandwidth requirements of designs in this design space. Finally, we present case studies enabled by LoopTree that show how exploring this larger space results in designs that require less buffer space (e.g., up to 7.6× buffer space reduction for the same off-chip transfers).