2026-09-19T04:59:53Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1283982026-06-17T14:46:31Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022

Russ Tedrake. Florence, Peter R.(Peter Raymond) Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2020-11-06T21:07:56Z 2020-11-06T21:07:56Z 2020 2020 https://hdl.handle.net/1721.1/128398 1202816736 This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2020 Cataloged from student-submitted PDF of thesis. Includes bibliographical references (pages 115-127). We would like to have highly useful robots which can richly perceive their world, semantically distinguish its fine details, and physically interact with it sufficiently for useful robotic manipulation. This is hard to achieve with previous methods: prior work has not equipped robots with the scalable ability to understand the dense visual state of their varied environments. The limitations have both been in the state representations used, and how to acquire them without significant human labeling effort. In this thesis we present work that leverages self-supervision, particularly via a mix of geometrical computer vision, deep visual learning, and robotic systems, to scalably produce dense visual inferences of the world state. These methods either enable robots to teach themselves dense visual models without human supervision, or they act as a large multiplying factor on the value of information provided by humans. Specifically, we develop a pipeline for providing ground truth labels of visual data in cluttered and multi-object scenes, we introduce the novel application of dense visual object descriptors to robotic manipulation and provide a fully robot-supervised pipeline to acquire them, and we leverage this dense visual understanding to efficiently learn new manipulation skills through imitation. With real robot hardware we demonstrate contact-rich tasks manipulating household objects, including generalizing across a class of objects, manipulating deformable objects, and manipulating a textureless symmetrical object, all with closed-loop, real-time vision-based manipulation policies. by Peter R. Florence. Ph.D. Ph.D. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science 141 pages eng Massachusetts Institute of Technology MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. Dense visual learning for robot manipulation Thesis application/pdf 2020-11-06T21:07:55Z Publication Doctoral EECS unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="c0d62092-bb72-4696-9a26-fe4b6f9d17ed"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>Dense visual learning for robot manipulation</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2020</PublicationDate> <Authors> <Author> <DisplayName>Florence, Peter R.(Peter Raymond)</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>We would like to have highly useful robots which can richly perceive their world, semantically distinguish its fine details, and physically interact with it sufficiently for useful robotic manipulation. This is hard to achieve with previous methods: prior work has not equipped robots with the scalable ability to understand the dense visual state of their varied environments. The limitations have both been in the state representations used, and how to acquire them without significant human labeling effort. In this thesis we present work that leverages self-supervision, particularly via a mix of geometrical computer vision, deep visual learning, and robotic systems, to scalably produce dense visual inferences of the world state. These methods either enable robots to teach themselves dense visual models without human supervision, or they act as a large multiplying factor on the value of information provided by humans. Specifically, we develop a pipeline for providing ground truth labels of visual data in cluttered and multi-object scenes, we introduce the novel application of dense visual object descriptors to robotic manipulation and provide a fully robot-supervised pipeline to acquire them, and we leverage this dense visual understanding to efficiently learn new manipulation skills through imitation. With real robot hardware we demonstrate contact-rich tasks manipulating household objects, including generalizing across a class of objects, manipulating deformable objects, and manipulating a textureless symmetrical object, all with closed-loop, real-time vision-based manipulation policies.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>