Making Hands: Neural Implicit Manifold Learning of Hand Gestures

Abstract

The human hand is a complex and sophisticated biological machine. Hand gesturing is key to our understanding of and interacting with the world around us. Hand gesturing is also instrumental in design and making. I argue that understanding the geometry of the space of hand gestures leads to an intuitive human-computer interaction. I decompose the gesture into its constituent parts, i.e., the hand motion – global coordinate system – and the hand pose – local coordinate system. I propose modeling the configuration space of hands as a high-dimensional manifold via neural unsigned distance fields, and I define plausible hand poses as points on the manifold. Next, I apply a distance metric to their configuration space. A trajectory in that space is a finite or infinite sequence of hand poses. These trajectories represent the different ways that the hand gestures. To demonstrate my approach, I restrict my study to a dataset of hands grasping everyday objects, and I evaluate my model on unknown grasps. Extending the model, the learned manifold acts as a prior for hand pose denoising, hand pose interpolation, and hand pose synthesis. Constraining that space can be interpreted as excluding impossible hand poses while constraining the manifold can be interpreted as defining a set of desirable hand poses. The former emphasizes the importance of bridging deep learning with existing mathematical structures, while the latter underlines future directions for the fields of design and computational making.