An exploration of strategies for discrete learning

Abstract

My research investigates the question of compositionality posed as an engineering question: How do we take two or more machines to build a compound machine, such that the compound machine has the combined abilities of its constituent parts, plus the abilities that arise from the cooperation between its parts? In order to illuminate this problem I have proposed a multi-level crossbar with bidirectional discrete wires as a substrate that allows for biologically plausible computation and communication, and candidate algorithms for a learning protocol between two connected agents. The protocol works locally, and enables the agents to learn a dataset of objects, while bootstrapping their own communication mechanism for transmitting information to each other. The system of the two agents, along with the local learning protocol, is equivalent to a discrete autoencoder working on unsupervised small data, with the goal of compression. It can be further generalized as a Programmable Logic Array analogous structure with uncertainty, i.e. incorporating "don't know" values to the existing true and false values, for which we can ask what possible functions can be learned locally. As a consequence of this work, I have learned what approaches don't work, and what approaches could be in the right direction, when it comes to doing discrete learning. I have documented my approaches, in terms of how successful they were, what were the main issues, what didn't work, what did, and what useful ideas can be extended on. I have also learned what is possible to be done by using only few discrete values.