Generalizable Robot Manipulation through Task and Motion Planning and Interactive Perception
Author(s)
Fang, Xiaolin
DownloadThesis PDF (18.63Mb)
Advisor
Kaelbling, Leslie Pack
Lozano-Pérez, Tomás
Terms of use
Metadata
Show full item recordAbstract
For a robot operating in a daily household environment, generality is of great importance. It should be able to generalize to different tasks that involve different objects in varying backgrounds and configurations.
In this thesis, we will move towards this goal from two perspectives. We will first present a strategy for designing a robot manipulation system that can generalize to a wide range of goals, environments, and objects. Such generality is achieved through task and motion planning with affordances estimated by both learned and engineered modules. We demonstrate that this strategy can enable a single policy to perform a wide variety of real-world manipulation tasks. Next, we will present an interactive perception solution to deal with the uncertainty in the estimated affordances, with a focus on the segmentation of objects. We adopt an object-based belief representation to estimate the uncertainty coming from predicted segmentation, and select actions to reduce that efficiently. Our experiments show that our system can generalize better to different environments and reduce uncertainty more efficiently compared to our baselines.
Date issued
2022-05Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer SciencePublisher
Massachusetts Institute of Technology