TEXterity: Tactile Extrinsic deXterity

Abstract

This thesis introduces the concept of TEXterity (Tactile Extrinsic deXterity) to address challenges in robotic manipulation. Focusing on tactile sensing, TEXterity aims to enhance dexterity by enabling robots to perceive and act upon extrinsic contact between the grasped object and the environment. Identifying interpretability, observability, and uncertainty as key challenges in tactile sensing, this thesis sets out to answer four pivotal questions: • Is tactile sensing actually useful?: Chapter 2 explores the practical utility of tactile sensing through an examination of its application in a peg-in-hole insertion task. It demonstrates the advantages of tactile sensing over conventional force-torque sensing. • How can we exploit tactile sensing efficiently?: Chapter 3 proposes an efficient approach to exploit tactile sensing by introducing extrinsic contact as an interpretable representation. This method offers scalability and effectiveness in utilizing tactile data. • How can we reason about extrinsic contact with tactile sensing?: Chapter 4 develops a novel method for simultaneous estimation and control of extrinsic contact states, addressing uncertainties introduced by physical interactions. It enables robots to reason effectively about extrinsic contact using tactile sensing in a controlled manner. • How can we achieve extrinsic dexterity with tactile sensing?: Chapter 5 extends the simultaneous estimation and control method to achieve extrinsic dexterity, showcasing precise in-hand sliding regrasps facilitated by pushing the object against the external environment. The conclusion summarizes the key findings, emphasizing the significance of tactile sensing and TEXterity in addressing challenges and advancing robotic manipulation. Strategies to tackle major challenges are outlined, focusing on interpretability, observability, and uncertainty. In essence, this thesis lays the groundwork for unlocking the potential of tactile sensing in robotic manipulation, offering insights, solutions, and avenues for future research to propel the field toward achieving TEXterity and further toward human-level dexterity. Please see real-world demonstration videos at this url: https://youtube.com/playlist?list=PLyqras4WFJI2q_TQshgVdWsKlxtjKolKL&si=AlSWgcu4BuJSkMCo