RF-based 3D skeletons
Name
rfpose3d-sigcomm-zhao.pdf
Description
Accepted version
Size
5.73 MB
Format
Adobe PDF
Checksum (MD5)
8e93180d5519a261f9d03b4d731e36ff
Author(s)
Zhao, Mingmin
Tian, Yonglong
Zhao, Hang
Alsheikh, Mohammad Abu
Li, Tianhong
Hristov, Rumen H.
Kabelac, Zachary E.
Katabi, Dina
Torralba, Antonio
Date Issued
August 2018
Journal
Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication
Publisher
ACM Press
Citation
Zhao, Mingmin et al. "RF-based 3D skeletons." Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, Budapest, Hungary, ACM Press, August 2018. © 2018 Association for Computing Machinery
Version
Author's final manuscript
Abstract
This paper introduces RF-Pose3D, the first system that infers 3D human skeletons from RF signals. It requires no sensors on the body, and works with multiple people and across walls and occlusions. Further, it generates dynamic skeletons that follow the people as they move, walk or sit. As such, RF-Pose3D provides a significant leap in RF-based sensing and enables new applications in gaming, healthcare, and smart homes. RF-Pose3D is based on a novel convolutional neural network (CNN) architecture that performs high-dimensional convolutions by decomposing them into low-dimensional operations. This property allows the network to efficiently condense the spatio-temporal information in RF signals. The network first zooms in on the individuals in the scene, and crops the RF signals reflected off each person. For each individual, it localizes and tracks their body parts - head, shoulders, arms, wrists, hip, knees, and feet. Our evaluation results show that RF-Pose3D tracks each keypoint on the human body with an average error of 4.2 cm, 4.0 cm, and 4.9 cm along the X, Y, and Z axes respectively. It maintains this accuracy even in the presence of multiple people, and in new environments that it has not seen in the training set. Demo videos are available at our website: http://rfpose3d.csail.mit.edu.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3230543.3230579
