Non-Line-of-Sight 3D Object Reconstruction via mmWave Surface Normal Estimation

Abstract

This paper presents the design, implementation, and evaluation of mmNorm, a new and highly-accurate method for non-line-of-sight 3D object reconstruction using millimeter wave (mmWave) signals. In contrast to past approaches for millimeter-wave-based imaging that perform backprojection for 3D object reconstruction, mmNorm reconstructs the surface by estimating the object’s surface normals. To do this, it introduces a novel algorithm that directly estimates the surface normal vector field from mmWave reflections. By then inverting the normal field, it can reconstruct structural isosurfaces, then solve for the exact surface through a novel mmWave optimization framework. We built an end-to-end prototype of mmNorm using a TI IWR1443 Boost mmWave radar and a UR5e Robotic Arm, and evaluated it in over 110 real-world experiments across more than 60 different everyday objects. In a head-to-head comparison with state-of-theart baselines, mmNorm achieves 96% reconstruction accuracy (3D F-score) compared to 78% for the best-performing baseline. These results show that mmNorm is capable of high-accuracy mmWave object reconstruction. The codebase and a video demonstration are available here: https://github.com/signalkinetics/mmNorm