Multi-bounce Returns for Specular Surface Mapping from Consumer-grade Flash LiDAR

Abstract

This thesis proposes an approach to leverage multi-bounce returns of a flash LiDAR on portable smartphones for 3D specular surface reconstruction. This is an important research problem as most traditional LiDAR systems fail to detect specular surfaces. As mirror and glass are everywhere, vision systems failing to detect specular surface can be detrimental. Applications like mapping may become inaccurate, and more critically, robots could crash into undetected windows during navigation, leading to potentially fatal outcomes. We perceive this work can impactfully enhance the robustness of specular surface detection, with LiDAR complementing any kind of vision system, particularly image-based.

Traditional LiDAR systems typically assume that all returns are single-bounce, which can lead to inaccurate representations of specular surfaces like mirrors or glass, often causing them to appear as though there is a hole. In contrast, this approach models the multi-bounce paths, providing a more accurate reconstruction of these specular surfaces.

We operate with a consumer-grade LiDAR that does not require manual calibration and can be operated in real-time on an affordable and portable smartphone. Consumer-grade LiDAR multi-beam flash LiDAR is challenging with its coarse resolution, co-located sensors, and multiplexing setup. In face of these challenges, we propose to solve the association problem with the `reciprocal pair’ algorithm, which can discern different types of bounces from the multi-bounce returns.

The algorithm is shown to detect over multiple consecutive frames for dense mirror mapping. In addition to 3D reconstruction, we show multi-bounce returns help to enhance performances on applications such as segmentation and novel view synthesis. Our method can be combined with these state-of-the-art learned-based model, enhancing its robustness by discerning ambiguous scenarios. In general, this approach can map various specular surfaces like mirrors and glasses, without making assumption about particular specular surface shapes, and can operate on non-perpendicular specular-diffuse surface pairs.