Reducing data movement in multicore chips with computation and data co-scheduling

Abstract

Energy efficiency is the main limitation to the performance of parallel systems. Current architectures often focus on making cores more efficient. However, data movement is much more costly than basic compute operations. For example, at 28 nm, a main memory access is 100x slower and consumes 1000x the energy of a floatingpoint operation, and moving 64 bytes across a 16-core processor is 50 x slower and consumes 20 x the energy. Without a drastic reduction in data movement, memory accesses and communication costs will limit the scalability of future computing systems. Conventional hardware-only and software-only techniques miss many opportunities to reduce data movement. This thesis presents computation and data co-scheduling (CDCS), a technique that jointly performs computation and data placement to reduce both on-chip and off-chip data movement. CDCS integrates hardware and software techniques: Hardware lets software control data mapping to physically distributed caches, and software uses this support to periodically reconfigure the chip, minimizing data movement. On a simulated 64-core system, CDCS outperforms a standard last-level cache by 46% on average (up to 76%) in weighted speedup, reduces both on-chip network traffic (by 11 x) and off-chip traffic (by 23%), and saves 36% of system energy.