High-resolution tactile sensing for reactive robotic manipulation

Abstract

This thesis explores tactile sensing to enable reactive behavior in robotic manipulation. More specifically, we focus on developing high-resolution vision-based tactile sensing hardware, perceptual algorithms, and controller designs for robotic manipulation. Tactile sensing plays a key role in human manipulation. However, the existing artificial tactile sensors have multiple limitations in terms of form factor, robustness, and sparse measurement. Tactile sensors are rarely integrated into the current robotic manipulation systems. In this thesis, we design new vision-based tactile sensors that capture the contact surface with high-resolution images and reconstruct the 3D geometry of the contact surface. We first design a variation of the GelSight sensor that improves the accuracy of the depth map reconstruction.

To further optimize the form factor and enhance the robustness, we designed another vision-based tactile sensor, GelSlim, which keeps the high-resolution sensing output but has a slimmer former, sharper tip, and improved robustness. Based on the new sensor, we propose algorithms to distill useful contact information from the raw signal output. The key challenge is connecting the contact geometry directly observed from the raw image to contact signals that have meanings in the context of contact mechanics, e.g., contact forces, contact slip. We use an algorithm to track the gel deformation and compare it with a rigid body motion to detect incipient slip. We deploy an inverse Finite Element Method (iFEM) to reconstruct the contact force distribution. Finally, we explore how the tactile signals can be fed into the control loop in real manipulation tasks. We choose 2 representative contact rich manipulation tasks that benefit from tactile control: cable following and object insertion.

We implement cable following by sensing & controlling both the state of the grasp of the cable and its configuration in realtime to allow smooth sliding of the fingers along a cable. We train a general tactile-based RL insertion policy in an end-to-end fashion to align the object pose with the insertion hole and keep sticking contact of the grasp by detecting incipient slip during the contact exploration. The RL insertion policy is capable of inserting novel objects, for which we show that tactile feedback is more informative than force-torque feedback.