Connecting symbols to primitive percepts using expectation as feedback

Abstract

This thesis is a step towards understanding and building mechanisms that connect symbols to primitive percepts. Symbols, such as those used in symbolic reasoning and language, are more intuitive to us humans since our languages and artifacts are highly symbolic. On the other hand, primitive percepts are results of processes that combine large amounts of evidence numerically, and are thus opaque to our human understanding. Inspired by an optical illusion known as Kanizsa's Triangle, I propose that expectation is essential to perception at every layer, including the very lowest levels. My proposal is that expectations generate hallucinations, that when mutually constrained with the sense data, produce a reasonable interpretation of that data. Furthermore, the mechanisms that project expectations may just be what connects, at an appropriate level, symbols to percepts. To engineer such mechanisms in an artificial machine, I start with feedback at every layer as a simple mechanism for projecting expectations. I then present a multi-layered distributed structure that incorporates such feedback expectations, and that recognizes and hallucinates images of digits using a bidirectional mutual constraining process. In this process the higher layers work as critics on lower layers, filling in details and removing noise to improve the data at that level.