SAL : a Self-Aware Learning system

Abstract

In this thesis, I take a step towards understanding how and why humans learn to solve problems about their solving of problems. I present a general-purpose neural reinforcement learning system called SAL, which can learn to think about its own problem solving, and use this capability to learn how to solve problems at another level. I show that SAL can use self-reference to articulate, and learn to articulate, its thoughts to a human, and internalize and apply a human's help, in natural language. I also demonstrate that SAL's abilities are enabled by an internal representation that shares important properties with, and is easily converted between, natural language. On the practical side, I argue that SAL can inform production question answering systems research. SAL can answer multi-step questions that are grounded in the world by extracting operational knowledge from pre-trained word embeddings. As an example, SAL knows how to use the action associated with \grab [the] diesel jug" to get closer to a solution, given the state of a physical world and a goal. And SAL can do this without any actual experience using (and without ever being told by a human about) any action associated with \grab" or the argument \diesel jug." SAL can do so with both very little training reward data and without assuming anything about the operational meaning of a particular lexical item, or composition of them, at first. Alternatively, typical neural reinforcement learning systems can not learn like SAL; they only work with a level of data that would be difficult to achieve in the real world. SAL's implementation, trained models, analysis code, and instructions, are at https://github.com/TristanThrush/sal. It is easy to add new problems (even in new domains) that you want SAL to learn.