Grounding language in spatial routines
Author(s)Tellex, Stefanie, 1980-
Massachusetts Institute of Technology. Dept. of Architecture. Program In Media Arts and Sciences
MetadataShow full item record
This thesis describes a spatial language understanding system based on a lexicon of words defined in terms of spatial routines. A spatial routine is a script composed from a set of primitive operations on sensor data, analogous to Ullman's visual routines. By finding a set of primitives that underlie natural spatial language, the meaning of spatial terms can be succinctly expressed in a way that can be used to obey natural language commands. This hypothesis is tested by using spatial routines to build a natural language interface to a real time strategy game, in which a player controls an army of units in a battle. The system understands the meaning of context-dependent natural language commands such as "Run back!" and "Move the marines on top above the fiamethrowers on the bottom." In evaluation, the system successfully interpreted a range of spatial commands not seen during implementation, and exceeded the performance of a baseline system. Beyond real-time strategy games, spatial routines may provide the basis for interpreting spatial language in a broad range of physically situated language understanding systems, such as mobile robots or other computer game genres.
Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2006.Includes bibliographical references (p. 105-108).
DepartmentMassachusetts Institute of Technology. Dept. of Architecture. Program In Media Arts and Sciences
Massachusetts Institute of Technology
Architecture. Program In Media Arts and Sciences