Contact-aware and multi-modal robotic manipulation
Name
zhao-alanzhao-phd-meche-2025-thesis.pdf
Description
Thesis PDF
Size
19.46 MB
Format
Adobe PDF
Checksum (MD5)
4d96cc975c0b42c7319751aa15fe5f1f
Author(s)
Zhao, Jialiang
Advisor(s)
Adelson, Edward H.
Date Issued
February 2025
Publisher
Massachusetts Institute of Technology
Abstract
Intelligent robotic manipulation has advanced significantly in recent years, driven by progress in foundational cognitive models, sensor-fusion techniques, and improvements in actuators and sensors. However, most contemporary robotic systems still lack the ability to effectively recognize and understand contact dynamics, which are critical for performing manipulation tasks beyond basic pick-and-place operations. This thesis argues and proves that contact awareness is essential for the successful deployment of robotic systems, not only in structured environments such as factories but also in unstructured settings like domestic households. Achieving contact awareness necessitates advancements in three key areas: the development of improved contact-sensing hardware, the creation of more expressive frameworks for representing and interpreting contact information, and the design of efficient modality-fusion algorithms to integrate these capabilities into robotic action planning. This work addresses these challenges by (1) proposing novel mechanical designs that enable touch sensors to adopt more compact and versatile forms while enhancing their durability and manufacturability, (2) introducing a foundational representation learning framework capable of learning a shared tactile latent representation, which can be transferred across different sensors and downstream tasks, and (3) developing a compositional diffusion-based approach for action prediction that integrates tactile sensing signals with other perception modalities, thereby enabling learning across diverse environments and promoting policy reuse. Along the way, this thesis demonstrates that tactile sensors can be both compact and versatile, challenging common perceptions to the contrary. It also establishes that tactile sensing is indispensable not only for high-precision tasks, such as electronics assembly, but also for everyday activities, including cooking and tool usage.
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
Persistent DSpace Link
