Enhancing Robotic Manipulation of Liquid Using a Digitally Fabricated Intelligent Wearable Device

Abstract

Despite recent exponential advances in computer vision and reinforcement learning, it remains challenging for robots to interact with liquids due to visual obstructions, transparent liquids, and fine-grained splashes. Yet, a substantial opportunity exists for robotics to excel in liquid identification and manipulation, given its potential role in chemical handling in laboratories and various manufacturing sectors such as pharmaceuticals or beverages. Recent advancements in electronic wearables, designed to replicate or surpass the functions and attributes of human skin, and their convergence with machine learning have provided opportunities to enhance the capabilities of robotic systems. Here, we present a novel approach for liquid class identification and position estimation with the robotic wearable device that can ‘see through’ the container, leveraging electrical impedance sensing. We design and mount a digitally embroidered electrode array to a commercial robotic gripper. Coupled with a customized impedance sensing board, we collect data on liquid manipulation with a swept frequency sensing mode and a frequency-specific impedance measuring mode. Our developed learning-based models achieve an accuracy of 93.33% in classifying 9 different types of liquids (8 liquids + air) and 97.65% in estimating the liquid position in the cup without any vision system present. We investigate the effectiveness of our system with a series of ablation studies. These findings highlight our work as a promising solution for enhancing robotic manipulation in liquid-related tasks.