State discovery for autonomous learning

Abstract

This thesis is devoted to the study of algorithms for early perceptual learning for an autonomous agent in the presence of feedback. In the framework of associative perceptual learning with indirect supervision, three learning techniques are examined in detail: * short-term on-line memory-based model learning; * long-term on-line distribution-based statistical estimation; * mixed on- and off-line continuous learning of gesture models. The three methods proceed within essentially the same framework, consisting of a perceptual sub-system and a sub-system that implements the associative mapping from perceptual categories to actions. The thesis contributes in several areas - it formulates the framework for solving incremental associative learning tasks; introduces the idea of incremental classification with utility, margin and boundary compression rules; develops a technique of sequence classification with Support Vector Machines; introduces an idea of weak transduction and offers an EM-based algorithm for solving it; proposes a mixed on- and off-line algorithm for learning continuous gesture with reward-based decomposition of the state space. The proposed framework facilitates the development of agents and human-computer interfaces that can be trained by a naive user. The work presented in this dissertation focuses on making these incremental learning algorithms practical.