Understanding Vision-based Dynamics Models
Name
Liu-cynliu98-meng-eecs-2021-thesis.pdf
Description
Thesis PDF
Size
17.5 MB
Format
Adobe PDF
Checksum (MD5)
278a55f93bf68f240944f5b77fd79964
Author(s)
Liu, Cynthia
Advisor(s)
Torralba, Antonio
Date Issued
June 2021
Publisher
Massachusetts Institute of Technology
Abstract
Recent developments in vision-based dynamics models have helped researchers achieve state-of-the-art results in a number of fields. For instance, in model-based reinforcement learning, vision-based methods perform extremely well on a variety of games and control tasks while using orders of magnitudes less data than model-free methods. One example is GameGAN, which learns to simulate the dynamics of observed games solely from visual and action inputs. However, there is very little understanding of these models and how they work. To address this lack of understanding, we apply the Network Dissection framework to analyze vision-based dynamics prediction models. We inspect individual trained neurons in convolutional layers of these models and modify the output of neurons to understand their effect on the representation. We also theoretically extend the Network Dissection framework by generalizing it to fully connected layers instead of only convolutional layers. Overall, we provide insight into the node-level workings of dynamics models.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
