Manipulation of unknown objects via contact configuration regulation

Abstract

In this thesis, we present an approach to robotic manipulation of unknown objects through the regulation of an object's contact configuration: the location, geometry, and mode of all contacts between the object, robot, and environment. A contact configuration constrains the forces and motions that can be applied to the object. As such, the ability to predict and regulate the contact configuration facilitates dexterous manipulation. With this as our guiding principle, we develop a joint estimation and control framework to reactively manipulate unknown objects in the gravity plane.

We begin by building a model to describe the interactions between a polygonal object, the robot, and the environment. This is accomplished by deriving the kinematic and wrench constraints associated with the geometric and frictional properties of each contact.

Our estimator generates the wrench constraints, the contact mode/geometry, and the object's shape/pose, using a combination of tactile and (limited) visual feedback. There are two separate modules: the friction estimator, which infers the friction constraints and contact mode from the measured force; and the kinematic estimator, which infers the contact geometry and the object's shape/pose from tactile and visual feedback.

The controller regulates the system's pose along the admissible motion directions, while simultaneously regulating the end-effector contact wrench to maintain the desired contact mode and geometry. The motion and wrench control objectives are balanced through a combination of a high-level controller, which synthesizes the kinematic and wrench constraints; and an impedance layer, which executes the motion.

We implement this estimation and control framework on our manipulation platform, and demonstrate that it allows the robot to reactively execute a wide variety of manipulation tasks. These include basic motions, like reorienting an object, sliding it along the ground, or performing a regrasp; as well as more advanced primitives, like using a wall as a support to reorient an object, or regulating the contact geometry between the object and the ground. Finally, we conduct ablation studies to understand the contributions from visual and tactile feedback in our manipulation framework.