Dynamics and learning in recurrent neural networks

Abstract

This thesis is a study of dynamics and learning in recurrent neural networks. Many computations of neural systems are carried out through a network of a large number of neurons. With massive feedback connections among these neurons, a study of its dynamics is necessary in order to understand the network's function. In this thesis, I aim at studying several recurrent network models and relating the dynamics with the networks' computation. For this purpose, three systems are studied and analyzed in detail: The first one is a network model for direction selectivity; the second one is a generalized network of Winner-Take-All; the third one is a model for integration in head-direction systems. One distinctive feature of neural systems is the ability of learning. The other part of my thesis is on learning in biologically motivated neural networks. Specifically, I study how the spike-time-dependent synaptic plasticity helps to stabilize persistent neural activities in the ocular motor integrator. I study the connections between back-propagation and contrastive-Hebbian learning, and show how backpropagation could be equivalently implemented by contrastive-Hebbian learning in a layered network. I also propose a learning rule governing synaptic plasticity in a network of spiking neurons and compare it with recent experimental results on spike-time-dependent plasticity.