Neuro-RAM unit with applications to similarity testing and compression in spiking neural networks
Name
LIPIcs-DISC-2017-33.pdf
Description
Published version
Size
866.93 KB
Format
Adobe PDF
Checksum (MD5)
3595f31b078063100be2e16b2d7cabe5
Author(s)
Lynch, Nancy A.
Parter, Merav
Musco, Cameron
Date Issued
2017
Citation
Lynch, Nancy, Parter, Merav and Musco, Cameron. 2017. "Neuro-RAM unit with applications to similarity testing and compression in spiking neural networks."
Version
Final published version
Abstract
© Nancy Lynch, Cameron Musco, and Merav Parter;. We study distributed algorithms implemented in a simplified biologically inspired model for stochastic spiking neural networks. We focus on tradeoffs between computation time and network complexity, along with the role of noise and randomness in efficient neural computation. It is widely accepted that neural spike responses, and neural computation in general, is inherently stochastic. In recent work, we explored how this stochasticity could be leveraged to solve the 'winner-take-all' leader election task. Here, we focus on using randomness in neural algorithms for similarity testing and compression. In the most basic setting, given two n-length patterns of firing neurons, we wish to distinguish if the patterns are equal or ϵ-far from equal. Randomization allows us to solve this task with a very compact network, using O (√n log n/ϵ) auxiliary neurons, which is sublinear in the input size. At the heart of our solution is the design of a t-round neural random access memory, or indexing network, which we call a neuro-RAM. This module can be implemented with O(n/t) auxiliary neurons and is useful in many applications beyond similarity testing - e.g., we discuss its application to compression via random projection. Using a VC dimension-based argument, we show that the tradeoff between runtime and network size in our neuro-RAM is near optimal. To the best of our knowledge, we are the first to apply these techniques to stochastic spiking networks. Our result has several implications - since our neuro-RAM can be implemented with deterministic threshold gates, it shows that, in contrast to similarity testing, randomness does not provide significant computational advantages for this problem. It also establishes a separation between feedforward networks whose gates spike with sigmoidal probabilities, and well-studied deterministic sigmoidal networks, whose gates output real number sigmoidal values, and which can implement a neuro-RAM much more efficiently.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.4230/LIPIcs.DISC.2017.33
