2026-09-20T03:44:19Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/770772022-01-13T07:54:29Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Nicholas Roy. Simeonov, Dimitar N Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2013-02-14T19:16:40Z 2013-02-14T19:16:40Z 2012 2012 http://hdl.handle.net/1721.1/77077 825771159 Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012. Cataloged from PDF version of thesis. Includes bibliographical references (p. 81-84). Natural language interaction can enable us to interface with robots such as the Personal Robot 2 (PR2), without the need for a special training or equipment. Programming such a robot to follow commands is challenging because natural language has a complex structure and semantics, a model for which needs to be based on linguistic knowledge or learned from examples. In this thesis we first enable the PR2 robot to follow manipulation commands expressed in natural language by applying the Generalized Grounding Graph (G3 ). We model the PR2's actions and their trajectories in the physical environment, define the state-action space and learn a grounding model from an annotated corpus of robot actions aligned with commands. We achieved lower overall performance than previous implementations of G3 had reported. After that, we present an approach for using the linguistic technique of coreference resolution to improve the robot's ability to understand commands consisting of multiple clauses. We constrain the groundings for coreferent phrases to be identical by merging their nodes in the grounding graph. We show that using coreference information increases the robot ability to infer the right action sequence. This brings the robotic capabilities of modeling and understanding natural language closer to our theoretical understanding of discourse. by Dimitar N. Simeonov. M.Eng. 84 p. eng Massachusetts Institute of Technology M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. The use of coreference resolution for understanding manipulation commands for the PR2 Robot Thesis application/pdf false Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="f4d2cf13-0e68-4dfb-b6dd-76072de94819"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>The use of coreference resolution for understanding manipulation commands for the PR2 Robot</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2012</PublicationDate> <Authors> <Author> <DisplayName>Simeonov, Dimitar N</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>Natural language interaction can enable us to interface with robots such as the Personal Robot 2 (PR2), without the need for a special training or equipment. Programming such a robot to follow commands is challenging because natural language has a complex structure and semantics, a model for which needs to be based on linguistic knowledge or learned from examples. In this thesis we first enable the PR2 robot to follow manipulation commands expressed in natural language by applying the Generalized Grounding Graph (G3 ). We model the PR2&apos;s actions and their trajectories in the physical environment, define the state-action space and learn a grounding model from an annotated corpus of robot actions aligned with commands. We achieved lower overall performance than previous implementations of G3 had reported. After that, we present an approach for using the linguistic technique of coreference resolution to improve the robot&apos;s ability to understand commands consisting of multiple clauses. We constrain the groundings for coreferent phrases to be identical by merging their nodes in the grounding graph. We show that using coreference information increases the robot ability to infer the right action sequence. This brings the robotic capabilities of modeling and understanding natural language closer to our theoretical understanding of discourse.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>