Characterizing and Optimizing Realistic Workloads on a Commercial Compute-in-SRAM Device
Name
3725843.3756132.pdf
Size
2.17 MB
Format
Adobe PDF
Checksum (MD5)
e55b38fcb6d4c4e78c5bb031012dca37
Author(s)
Zhang, Niansong
Zhu, Wenbo
Golden, Courtney
Ilan, Dan
Chen, Hongzheng
Batten, Christopher
Zhang, Zhiru
Date Issued
October 17, 2025
Publisher
ACM|58th IEEE/ACM International Symposium on Microarchitecture
Citation
Niansong Zhang, Wenbo Zhu, Courtney Golden, Dan Ilan, Hongzheng Chen, Christopher Batten, and Zhiru Zhang. 2025. Characterizing and Optimizing Realistic Workloads on a Commercial Compute-in-SRAM Device. In Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture (MICRO '25). Association for Computing Machinery, New York, NY, USA, 1011–1025.
Version
Final published version
Abstract
Compute-in-SRAM architectures offer a promising approach to
achieving higher performance and energy efficiency across a range
of data-intensive applications. However, prior evaluations have
largely relied on simulators or small prototypes, limiting the understanding of their real-world potential. In this work, we present
a comprehensive performance and energy characterization of a
commercial compute-in-SRAM device, the GSI APU, under realistic
workloads. We compare the GSI APU against established architectures, including CPUs and GPUs, to quantify its energy efficiency
and performance potential. We introduce an analytical framework
for general-purpose compute-in-SRAM devices that reveals fundamental optimization principles by modeling performance trade-offs,
thereby guiding program optimizations.
Exploiting the fine-grained parallelism of tightly integrated
memory-compute architectures requires careful data management.
We address this by proposing three optimizations: communicationaware reduction mapping, coalesced DMA, and broadcast-friendly
data layouts. When applied to retrieval-augmented generation
(RAG) over large corpora (10GB–200GB), these optimizations enable
our compute-in-SRAM system to accelerate retrieval by 4.8×–6.6×
over an optimized CPU baseline, improving end-to-end RAG latency by 1.1×–1.8×. The shared off-chip memory bandwidth is
modeled using a simulated HBM, while all other components are
measured on the real compute-in-SRAM device. Critically, this system matches the performance of an NVIDIA A6000 GPU for RAG
while being significantly more energy-efficient (54.4×-117.9× reduction). These findings validate the viability of compute-in-SRAM
for complex, real-world applications and provide guidance for advancing the technology.
Description
MICRO ’25, Seoul, Republic of Korea
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3725843.3756132
