The anatomy of visual pattern acquisition in deep learning

Abstract

Conventional wisdom says that convolutional neural networks use their filter hierarchy to learn feature structures. Yet very little work has been done to understand how the deep networks acquire those complicated features. Moreover, this blind spot opens up neural networks to nefarious security threats like data poisoning based backdoor attacks [Gu et al., 2017]. In this thesis, we study the question of how neural networks acquire particular visual patterns and how that behavior evolves with the learning rate. We aim to understand which patterns the neural network learns and how it prioritizes different patterns over one another over the learning process. We use a backdoor attack-based toolkit to analyze the learning process of the network and identify two fundamental properties of patterns that determine their fate: how often they appear in the dataset, and how strongly they correlate with particular labels. We empirically show how such properties determine how fast the network acquires patterns and what weight it puts on them. Finally, we propose a hypothesis that ties the pattern learning with the gradient from the network, and conclude by presenting a couple of experiments to support our claim.