2026-09-19T20:11:19Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1473512023-01-20T03:43:11Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022
Rodriguez, Alberto
Bauza Villalonga, Maria
Massachusetts Institute of Technology. Department of Mechanical Engineering
2023-01-19T18:47:29Z
2023-01-19T18:47:29Z
2022-09
2022-10-05T13:47:13.601Z
https://hdl.handle.net/1721.1/147351
In this thesis, we develop visuo-tactile perception to enable general and precise robotic manipulation. In particular, we want to study how to effectively process visual and tactile information to allow robots to expand their capabilities while remaining accurate and reliable.
We begin our work by focusing on developing tools for tactile perception. For the task of grasping, we use tactile observations to assess and improve grasp stability. Tactile information also allows extracting geometric information from contacts which is a task-independent feature. By learning to map tactile observations to contact shapes, we show robots can reconstruct accurate 3D models of objects, which can later be used for pose estimation.
We build on the idea of using geometric information from contacts by developing tools that accurately render contact geometry in simulation. This enables us to develop a probabilistic approach to pose estimation for novel objects based on matching real visuo-tactile observations to a set of simulated ones. As a result, our method does not rely on real data and yields accurate pose distributions.
Finally, we demonstrate how this approach to perception enables precise manipulations. In particular, we consider the task of precise pick-and-place of novel objects. Combining perception with task-aware planning, we build a robotic system that identifies in simulation which object grasps will facilitate grasping, planning, and perception; and selects the best one during execution. Our approach adapts to new objects by learning object-dependent models purely in simulation, allowing a robot to manipulate new objects successfully and perform highly accurate placements.
Ph.D.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright MIT
http://rightsstatements.org/page/InC-EDU/1.0/
Visuo-Tactile Perception for Dexterous Robotic Manipulation
Thesis
application/pdf
Doctoral
Doctor of Philosophy
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="53647499-240b-4a08-982e-edbdf45f4fb7">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Visuo-Tactile Perception for Dexterous Robotic Manipulation</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2022-09</PublicationDate>
<Authors>
<Author>
<DisplayName>Bauza Villalonga, Maria</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>In this thesis, we develop visuo-tactile perception to enable general and precise robotic manipulation. In particular, we want to study how to effectively process visual and tactile information to allow robots to expand their capabilities while remaining accurate and reliable.
We begin our work by focusing on developing tools for tactile perception. For the task of grasping, we use tactile observations to assess and improve grasp stability. Tactile information also allows extracting geometric information from contacts which is a task-independent feature. By learning to map tactile observations to contact shapes, we show robots can reconstruct accurate 3D models of objects, which can later be used for pose estimation.
We build on the idea of using geometric information from contacts by developing tools that accurately render contact geometry in simulation. This enables us to develop a probabilistic approach to pose estimation for novel objects based on matching real visuo-tactile observations to a set of simulated ones. As a result, our method does not rely on real data and yields accurate pose distributions.
Finally, we demonstrate how this approach to perception enables precise manipulations. In particular, we consider the task of precise pick-and-place of novel objects. Combining perception with task-aware planning, we build a robotic system that identifies in simulation which object grasps will facilitate grasping, planning, and perception; and selects the best one during execution. Our approach adapts to new objects by learning object-dependent models purely in simulation, allowing a robot to manipulate new objects successfully and perform highly accurate placements.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>