Responsive Space Environments: New Paradigms in Blending Virtual and Physical Exploration through Human-Robot Operations

Abstract

As humanity wanders further into space, the integration of robotics and virtual environments in space systems becomes increasingly pivotal, augmenting our capacity to explore, interact, and study remote environments. This research delves into the use cases, benefits, and challenges associated with Human-robot operations in virtual space analog environments, proposing innovative visualizations that work in tandem with automation to address challenges in mission planning and control, leveraging techniques akin to video game interfaces. In the pursuit of modeling and capturing the essence of distant environments, this research embarks on an investigation of cutting-edge 3D reconstruction techniques, expertly combined with high-definition rendering pipelines, to synthesize virtual environments sourced from real-world information and examine their role in planning and executing planetary exploration missions. This synthesis process becomes especially critical in remote settings, like on the Moon, Mars, and asteroids, where data transmission costs are high and efficiency is paramount in space exploration.

Accordingly, this dissertation also overviews the design and analyzes the performance of AKALL (Azure Kinect à la Luna), a software application for 3D imaging developed during this research that underwent rigorous testing on the SSERVI Lunar regolith testbed at NASA Ames Research Center. This software is designed to operate within an isolated Docker container and was integrated to operate a repurposed commercial imaging payload within Lunar Outpost’s Mobile Autonomous Prospecting Platform (MAPP) lunar rover, an integral component of the upcoming Intuitive Machines mission, (IM-2), slated to land on the Moon’s south pole early in 2024.