Discovering the patterns of human-environment interactions using scalable functional textiles

Abstract

Humans perform complex tasks in the real world thanks to rich and constant tactile perceptual input. Being able to record such tactile data would allow scientists from various disciplines to study human activities more fundamentally and quantitatively. Moreover, capturing large and diverse datasets on human-environment interactions and coupling them with machine learning models would allow the development of future intelligent robotic systems that mimic human behavior. Here, we present a textile-based tactile learning platform that enables researchers to record, monitor, and learn human activities and the associated interactions. Realized with inexpensive piezoresistive fibers (0.2 USD/m) and automated machine knitting, our functional textiles offer dense coverage (> 1000 sensors) over large complex surfaces (> 2000 cm2). Further, we leverage machine learning for sensing correction, ensuring that our system is robust against potential variations from individual receptors. To validate the capability of our sensor, we capture diverse human-environment interactions (> 1,000,000 tactile frames). We demonstrate that machine learning techniques can be used with our data to classify human activities, predict whole-body poses, and discover novel motion signatures. This work opens up new possibilities in wearable electronics, healthcare, manufacturing, and robotics.