Interactively designing robots in mixed reality using gestural control

Abstract

The demand for automation and robots is rapidly increasing across many domains which implicitly requires a simple and robust design process. The current workflow usually consists of designing and simulating robots on a computer which are then attempted to be recreated in the real-world. However, this method abstracts away important attributes about the physical terrain which can generate inaccurate simulations. This workflow also causes a linear bottleneck as designers have to iterate between their digital simulations and the real-world environment many times which is an inefficient use of time, money, and resources. We propose a new workflow and our contributions for a mixed-reality tool to design, simulate, and produce a class of link-based robots with ground locomotion in the future. Users first scan the realworld environment for the deployment of the desired robot. This process is executed through a mixed-reality headset and the layout is quickly registered on the computer. After the landscape is established, users can begin augmented robot creation with their own hands as the system is trained to track gestural control. Intuitive motions allow for fluid assembly and modifications. Throughout the design process, our work provides the capability of connecting to software to optimize robots for the given terrain. Once users are satisfied with the final design, they are ready to fabricate and assemble the final robot. This workflow is interactive, user-friendly, and ensures the robot is properly equipped for its deployed terrain.