Understanding computation through low-dimensional dynamics with recurrent neural networks

Abstract

One way to understand the brain is in terms of the computations it performs that allow an organism to survive in the world. Models of cognition and behavior can be useful for describing the computations that might be performed, but often provide little insight into how they are realized in neural network models. Addressing this disconnect requires tools for better understanding neural representations and how they are used for cognitive computations. Here, I present work towards developing a dynamical systems framework for neural computation, using a recurrent neural network (RNN) model.

To begin, I propose an analysis-by-synthesis method that uses local constraints on population activity to create RNNs that can solve a task. I demonstrate this method by creating networks that implement variations of a ring attractor, a classic model of representation in neural circuits. The first variation produces a specific drift-diffusion process over the attractor, similar to that proposed in a working memory model. As a second variation, I introduce an input that controls the speed of the network’s dynamics, creating a model for how contextual inputs can enable flexible behavior. Third, I explore ring attractor networks with dynamics of varying complexity.

Next, I provide a more detailed analysis of the relationship between neural representations and network connectivity. Again using a network synthesis technique, I show that RNNs with a wide variety of synaptic weight configurations can produce nearly identical ring attractors. To define the theoretical boundaries of the space containing all such networks, I identify underlying sources of variability as well as common features. In doing so, I develop a framework for relating the geometry and dynamics of constrained network states to the features of network connectivity.