Long Live TIME: Improving Lifetime for Training-In-Memory Engines by Structured Gradient Sparsification
Name
DAC18_251.pdf
Description
Accepted version
Size
673.06 KB
Format
Adobe PDF
Checksum (MD5)
57ac7850e72e3bf0a1d3f10fa1faaa0b
Author(s)
Han, Song
Date Issued
June 2018
Journal
2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Cai, Yi et al. “Long Live TIME: Improving Lifetime for Training-In-Memory Engines by Structured Gradient Sparsification.” Paper in the Proceedings of the 55th Annual Design Automation Conference, DAC ’18, San Francisco, CA, June 24-29, 2018, ACM © 2018 The Author(s)
Version
Author's final manuscript
Abstract
Deeper and larger Neural Networks (NNs) have made breakthroughsin many fields. While conventional CMOS-based computing plat-forms are hard to achieve higher energy efficiency. RRAM-basedsystems provide a promising solution to build efficient Training-In-Memory Engines (TIME). While the endurance of RRAM cells islimited, it’s a severe issue as the weights of NN always need to beupdated for thousands to millions of times during training. Gradi-ent sparsification can address this problem by dropping off mostof the smaller gradients but introduce unacceptable computationcost. We proposed an effective framework, SGS-ARS, includingStructured Gradient Sparsification (SGS) and Aging-aware RowSwapping (ARS) scheme, to guarantee write balance across wholeRRAM crossbars and prolong the lifetime of TIME. Our experi-ments demonstrate that 356×lifetime extension is achieved whenTIME is programmed to train ResNet-50 on Imagenet dataset withour SGS-ARS framework.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/DAC.2018.8465850
