Neural Network Training With Asymmetric Crosspoint Elements
Name
frai-05-891624.pdf
Description
Published version
Size
1.92 MB
Format
Adobe PDF
Checksum (MD5)
5163faeb5b5d1af287f8f831d35e2156
Author(s)
Onen, Murat
Gokmen, Tayfun
Todorov, Teodor K
Nowicki, Tomasz
del Alamo, Jesús A
Rozen, John
Haensch, Wilfried
Kim, Seyoung
Date Issued
2022
Journal
Frontiers in Artificial Intelligence
Publisher
Frontiers Media SA
Citation
Onen, Murat, Gokmen, Tayfun, Todorov, Teodor K, Nowicki, Tomasz, del Alamo, Jesús A et al. 2022. "Neural Network Training With Asymmetric Crosspoint Elements." Frontiers in Artificial Intelligence, 5.
Version
Final published version
Abstract
Analog crossbar arrays comprising programmable non-volatile resistors are under intense investigation for acceleration of deep neural network training. However, the ubiquitous asymmetric conductance modulation of practical resistive devices critically degrades the classification performance of networks trained with conventional algorithms. Here we first describe the fundamental reasons behind this incompatibility. Then, we explain the theoretical underpinnings of a novel fully-parallel training algorithm that is compatible with asymmetric crosspoint elements. By establishing a powerful analogy with classical mechanics, we explain how device asymmetry can be exploited as a useful feature for analog deep learning processors. Instead of conventionally tuning weights in the direction of the error function gradient, network parameters can be programmed to successfully minimize the total energy (Hamiltonian) of the system that incorporates the effects of device asymmetry. Our technique enables immediate realization of analog deep learning accelerators based on readily available device technologies.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.3389/frai.2022.891624
