FIP: Endowing Robust Motion Capture on Daily Garment by Fusing Flex and Inertial Sensors
Name
3706598.3714140.pdf
Size
27.63 MB
Format
Adobe PDF
Checksum (MD5)
0b8c4892708518df04597480e35a260f
Author(s)
Zheng, Ruonan
Fang, Jiawei
Yao, Yuan
Gao, Xiaoxia
Zuo, Chengxu
Guo, Shihui
Luo, Yiyue
Date Issued
April 25, 2025
Publisher
ACM|CHI Conference on Human Factors in Computing Systems
Citation
Ruonan Zheng, Jiawei Fang, Yuan Yao, Xiaoxia Gao, Chengxu Zuo, Shihui Guo, and Yiyue Luo. 2025. FIP: Endowing Robust Motion Capture on Daily Garment by Fusing Flex and Inertial Sensors. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). Association for Computing Machinery, New York, NY, USA, Article 927, 1–21.
Version
Final published version
Abstract
What if our clothes could capture our body motion accurately? This paper introduces Flexible Inertial Poser (FIP), a novel motion-capturing system using daily garments with two elbow-attached flex sensors and four Inertial Measurement Units (IMUs). To address the inevitable sensor displacements in loose wearables which degrade joint tracking accuracy significantly, we identify the distinct characteristics of the flex and inertial sensor displacements and develop a Displacement Latent Diffusion Model and a Physics-informed Calibrator to compensate for sensor displacements based on such observations, resulting in a substantial improvement in motion capture accuracy. We also introduce a Pose Fusion Predictor to enhance multimodal sensor fusion. Extensive experiments demonstrate that our method achieves robust performance across varying body shapes and motions, significantly outperforming SOTA IMU approaches with a 19.5% improvement in angular error, a 26.4% improvement in elbow angular error, and a 30.1% improvement in positional error. FIP opens up opportunities for ubiquitous human-computer interactions and diverse interactive applications such as Metaverse, rehabilitation, and fitness analysis. Our project page can be seen at Flexible Inertial Poser.
Description
CHI ’25, Yokohama, Japan
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3706598.3714140
