Grounding Verbs of Motion in Natural Language Commands to Robots
Name
Grounding_Verbs_of_Motion_in_Natural_Language_Comm.pdf
Size
2.02 MB
Format
Adobe PDF
Checksum (MD5)
a70a9228c775d25f55f60750cf27d170
Author(s)
Kollar, Thomas Fleming
Tellex, Stefanie A
Roy, Deb K
Roy, Nicholas
Date Issued
2014
Journal
Experimental Robotics
Publisher
Springer Nature
Citation
Kollar, Thomas et al. “Grounding Verbs of Motion in Natural Language Commands to Robots.” edited by O. Khatib, V. Kumar and G. Sukhatme. Experimental Robotics (2014): 31–47 © 2014 Springer-Verlag Berlin Heidelberg
Version
Author's final manuscript
Abstract
To be useful teammates to human partners, robots must be able to follow spoken instructions given in natural language. An important class of instructions involve interacting with people, such as “Follow the person to the kitchen” or “Meet the person at the elevators.” These instructions require that the robot fluidly react to changes in the environment, not simply follow a pre-computed plan. We present an algorithm for understanding natural language commands with three components. First, we create a cost function that scores the language according to how well it matches a candidate plan in the environment, defined as the log-likelihood of the plan given the command. Components of the cost function include novel models for the meanings of motion verbs such as “follow,” “meet,” and “avoid,” as well as spatial relations such as “to” and landmark phrases such as “the kitchen.” Second, an inference method uses this cost function to perform forward search, finding a plan that matches the natural language command. Third, a high-level controller repeatedly calls the inference method at each timestep to compute a new plan in response to changes in the environment such as the movement of the human partner or other people in the scene. When a command consists of more than a single task, the controller switches to the next task when an earlier one is satisfied. We evaluate our approach on a set of example tasks that require the ability to follow both simple and complex natural language commands. Keywords: Cost Function; Spatial Relation; State Sequence; Edit Distance; Statistical Machine Translation
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/978-3-642-28572-1_3
