Data-Efficient Learning for Complex and Real-Time Physical Problem Solving Using Augmented Simulation
Name
2011.07193.pdf
Description
Accepted version
Size
1.16 MB
Format
Adobe PDF
Checksum (MD5)
8f9f29201e00ecc7e0298b68e02aa1f3
Author(s)
Ota, Kei
Jha, Devesh K
Romeres, Diego
van Baar, Jeroen
Smith, Kevin A
Semitsu, Takayuki
Oiki, Tomoaki
Sullivan, Alan
Nikovski, Daniel
Tenenbaum, Joshua B
Date Issued
2021
Journal
IEEE Robotics and Automation Letters
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Ota, Kei, Jha, Devesh K, Romeres, Diego, van Baar, Jeroen, Smith, Kevin A et al. 2021. "Data-Efficient Learning for Complex and Real-Time Physical Problem Solving Using Augmented Simulation." IEEE Robotics and Automation Letters, 6 (2).
Version
Author's final manuscript
Abstract
Humans quickly solve tasks in novel systems
with complex dynamics, without requiring much interaction.
While deep reinforcement learning algorithms have achieved
tremendous success in many complex tasks, these algorithms
need a large number of samples to learn meaningful policies.
In this paper, we present a task for navigating a marble to the
center of a circular maze. While this system is very intuitive
and easy for humans to solve, it can be very difficult and
inefficient for standard reinforcement learning algorithms to
learn meaningful policies. We present a model that learns to
move a marble in the complex environment within minutes of
interacting with the real system. Learning consists of initializing
a physics engine with parameters estimated using data from
the real system. The error in the physics engine is then
corrected using Gaussian process regression, which is used
to model the residual between real observations and physics
engine simulations. The physics engine augmented with the
residual model is then used to control the marble in the maze
environment using a model-predictive feedback over a receding
horizon. To the best of our knowledge, this is the first time that
a hybrid model consisting of a full physics engine along with
a statistical function approximator has been used to control a
complex physical system in real-time using nonlinear modelpredictive control (NMPC).
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Center for Brains, Minds, and Machines
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/LRA.2021.3068887
