Representation Learning for Grounded Spatial Reasoning
Name
tacl_a_00004.pdf
Description
Published version
Size
8.79 MB
Format
Adobe PDF
Checksum (MD5)
07dd8216ba79e6339eb81fae10887bec
Author(s)
Janner, Michael
Narasimhan, Karthik
Barzilay, Regina
Date Issued
2018
Journal
Transactions of the Association for Computational Linguistics
Publisher
MIT Press - Journals
Version
Final published version
Abstract
The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards. The proposed model learns a representation of the world steered by instruction text. This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions. We train our model with reinforcement learning using a variant of generalized value iteration. The model outperforms state-of-the-art approaches on several metrics, yielding a 45% reduction in goal localization error.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1162/TACL_A_00004
