Spike-Based Winner-Take-All Computation: Fundamental Limits and Order-Optimal Circuits

Abstract

© 2019 Massachusetts Institute of Technology. Winner-take-all (WTA) refers to the neural operation that selects a (typ-ically small) group of neurons from a large neuron pool. It is conjectured to underlie many of the brain’s fundamental computational abilities. However, not much is known about the robustness of a spike-based WTA network to the inherent randomness of the input spike trains. In this work, we consider a spike-based k–WTA model wherein n randomly generated input spike trains compete with each other based on their underlying firing rates and k winners are supposed to be selected. We slot the time evenly with each time slot of length 1 ms and model the n input spike trains as n independent Bernoulli processes. We analytically characterize the minimum waiting time needed so that a target minimax decision accuracy (success probability) can be reached. We first derive an information-theoretic lower bound on the waiting time. We show that to guarantee a (minimax) decision error ≤ δ (where ≤ δ (0, 1)), the waiting time of any WTA circuit is at least ((1 − δ) log(k(n − k) + 1) − 1)TR, where R c (0, 1) is a finite set of rates and TR is a difficulty parameter of a WTA task with respect to set R for independent input spike trains. Additionally, TR is independent of δ, n, and k. We then design a simple WTA circuit whose waiting time is (Formula presented) 1 O log + log k(n − k) TR, δ provided that the local memory of each output neuron is sufficiently long. It turns out that for any fixed δ, this decision time is order-optimal (i.e., it matches the above lower bound up to a multiplicative constant factor) in terms of its scaling in n, k, and TR.