Online optimization with energy based models
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Leslie P. Kaelbling, Tomas Lozano-Perez and Joshua B. Tenenbaum.
MetadataShow full item record
This thesis examines the power of applying optimization on learned neural networks, referred to as Energy Based Models (EBMs). We first present methods that enable scalable training of EBMs, allowing an optimization procedure to generate high resolution images. We simultaneously show that resultant models are robust, compositional, and are further easy to learn online. Next we showcase how this optimization procedure can also be used to formulate plans in interactive environments. We further showcase how a similar procedure can be used to learn neural energy functions for proteins, enabling structural recovery through optimization. Finally, we show that by defining generation as a optimization procedure, we can combine generative models from different domains together, and apply optimization on the joint model. We show that this allows us to apply various logical operations on images generation, as well as learn to generate new concepts in a continual manner.
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020Cataloged from the official PDF of thesis.Includes bibliographical references (pages 69-78).
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Electrical Engineering and Computer Science.