Understanding Neural Networks from Theoretical and Biological Perspectives

Abstract

Neural Networks is an important subject of both biological and artificial intelligence, which we have not yet fully understand. Recently the field of neural networks has been popularized by the advancement of Deep Learning, and people start to know how to make artificial neural networks perform extrordinarily well in practical applications, rivaling human performance in many tasks. However, the extreme pursuits of practical performances are not underpinned by deep understanding, leading to an unbalanced and incomplete, if not precarious, field of intelligence.

Our theoretical and biological understanding about neural networks have fallen far behind their unprecedented proliferation. This creates an awkward situation — we create very capable intelligent systems that we do not understand, compromising our ability to predict, control and correctly treat such systems. Furthermore, these developments also do not directly translate to our understanding of the general concept of intelligence, including, most importantly, our own human intelligence.

With alleviating such issues as one of our goals, my colleagues and I spent a few years on trying to understand neural networks from both theoretical and biological perspectives, gaining interesting insights and results on these topics. This led to a few published papers that I compile to constitute my thesis. Finally, I discuss my preferred future directions of research under the current trends of AI/AGI.