An Intelligence Architecture for Grounded Language Communication with Field Robots
Name
Vol2_17.pdf
Description
Published version
Size
18.78 MB
Format
Adobe PDF
Checksum (MD5)
d6c1caba20047567fd1e68e271dcef62
Author(s)
Howard, Thomas
Stump, Ethan
Fink, Jonathan
Arkin, Jacob
Paul, Rohan
Park, Daehyung
Roy, Subhro
Barber, Daniel
Bendell, Rhyse
Schmeckpeper, Karl
Date Issued
2022
Journal
Field Robotics
Publisher
Field Robotics Publication Society
Citation
Howard, Thomas, Stump, Ethan, Fink, Jonathan, Arkin, Jacob, Paul, Rohan et al. 2022. "An Intelligence Architecture for Grounded Language Communication with Field Robots." Field Robotics, 2 (1).
Version
Final published version
Abstract
For humans and robots to collaborate effectively as teammates in unstructured environments, robots must be able to construct semantically rich models of the environment, communicate efficiently with teammates, and perform sequences of tasks robustly with minimal human intervention, as direct human guidance may be infrequent and/or intermittent. Contemporary architectures for human-robot interaction often rely on engineered human-interface devices or structured languages that require extensive prior training and inherently limit the kinds of information that humans and robots can communicate. Natural language, particularly when situated with a visual representation of the robot’s environment, allows humans and robots to exchange information about abstract goals, specific actions, and/or properties of the environment quickly and effectively. In addition, it serves as a mechanism to resolve inconsistencies in the mental models of the environment across the human-robot team. This article details a novel intelligence architecture that exploits a centralized representation of the environment to perform complex tasks in unstructured environments. The centralized environment model is informed by a visual perception pipeline, declarative knowledge, deliberate interactive estimation, and a multimodal interface. The language pipeline also exploits proactive symbol grounding to resolve uncertainty in ambiguous statements through inverse semantics. A series of experiments on three different, unmanned ground vehicles demonstrates the utility of this architecture through its robust ability to perform language-guided spatial navigation, mobile manipulation, and bidirectional communication with human operators. Experimental results give examples of component-level behaviors and overall system performance that guide a discussion on observed performance and opportunities for future innovation.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.55417/FR.2022017
