Augmented Reality Driving Using Semantic Geo-Registration
Author(s)Villamil, Ryan; Samarasekera, Supun; Chiu, Han-Pang; Murali, Varun; Munoz Kessler, Rodrigo Arturo; Kumar, Rakesh; ... Show more Show less
MetadataShow full item record
We propose a new approach that utilizes semantic information to register 2D monocular video frames to the world using 3D geo-referenced data, for augmented reality driving applications. The geo-registration process uses our predicted vehicle pose to generate a rendered depth map for each frame, allowing 3D graphics to be convincingly blended with the real world view. We also estimate absolute depth values for dynamic objects, up to 120 meters, based on the rendered depth map and update the rendered depth map to reflect scene changes over time. This process also creates opportunistic global heading measurements, which are fused with other sensors, to improve estimates of the 6 degrees-of-freedom global pose of the vehicle over state-of-the-art outdoor augmented reality systems [5, 18, 19]. We evaluate the navigation accuracy and depth map quality of our system on a driving vehicle within various large-scale environments for producing realistic augmentations.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Aeronautics and Astronautics; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)
Institute of Electrical and Electronics Engineers (IEEE)
Chiu, Han-Pang, Varun Murali, Ryan Villamil, G. Drew Kessler, Supun Samarasekera, and Rakesh Kumar. “Augmented Reality Driving Using Semantic Geo-Registration.” 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (March 2018).
Author's final manuscript