Show simple item record

dc.contributor.authorFazeli, Nima
dc.contributor.authorOller Oliveras, Miquel
dc.contributor.authorWu, J.
dc.contributor.authorWu, Z.
dc.contributor.authorTenenbaum, Joshua B
dc.contributor.authorRodriguez, A.
dc.date.accessioned2020-08-18T20:22:17Z
dc.date.available2020-08-18T20:22:17Z
dc.date.issued2019-01
dc.identifier.issn2470-9476
dc.identifier.urihttps://hdl.handle.net/1721.1/126656
dc.description.abstractHumans are able to seamlessly integrate tactile and visual stimuli with their intuitions to explore and execute complex manipulation skills. They not only see but also feel their actions. Most current robotic learning methodologies exploit recent progress in computer vision and deep learning to acquire data-hungry pixel-to-action policies. These methodologies do not exploit intuitive latent structure in physics or tactile signatures. Tactile reasoning is omnipresent in the animal kingdom, yet it is underdeveloped in robotic manipulation. Tactile stimuli are only acquired through invasive interaction, and interpretation of the data stream together with visual stimuli is challenging. Here, we propose a methodology to emulate hierarchical reasoning and multisensory fusion in a robot that learns to play Jenga, a complex game that requires physical interaction to be played effectively. The game mechanics were formulated as a generative process using a temporal hierarchical Bayesian model, with representations for both behavioral archetypes and noisy block states. This model captured descriptive latent structures, and the robot learned probabilistic models of these relationships in force and visual domains through a short exploration phase. Once learned, the robot used this representation to infer block behavior patterns and states as it played the game. Using its inferred beliefs, the robot adjusted its behavior with respect to both its current actions and its game strategy, similar to the way humans play the game. We evaluated the performance of the approach against three standard baselines and show its fidelity on a real-world implementation of the game.en_US
dc.language.isoen
dc.publisherAmerican Association for the Advancement of Science (AAAS)en_US
dc.relation.isversionof10.1126/scirobotics.aav3123en_US
dc.rightsCreative Commons Attribution-Noncommercial-Share Alikeen_US
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/4.0/en_US
dc.sourceMIT web domainen_US
dc.titleSee, feel, act: hierarchical learning for complex manipulation skills with multisensory fusionen_US
dc.typeArticleen_US
dc.identifier.citationFazeli, N. et al. "See, feel, act: hierarchical learning for complex manipulation skills with multisensory fusion." Science Robotics 4, 26 (January 2019): eaav3123 ©2019 Author(s)en_US
dc.contributor.departmentMassachusetts Institute of Technology. Department of Mechanical Engineeringen_US
dc.contributor.departmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratoryen_US
dc.relation.journalScience Roboticsen_US
dc.eprint.versionAuthor's final manuscripten_US
dc.type.urihttp://purl.org/eprint/type/JournalArticleen_US
eprint.statushttp://purl.org/eprint/status/PeerRevieweden_US
dc.date.updated2019-10-08T16:16:31Z
dspace.date.submission2019-10-08T16:16:36Z
mit.journal.volume4en_US
mit.journal.issue26en_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record