Neural physics simulation through volumetric reconstruction

Abstract

In this thesis we propose a method for simulating 3D object motion given 2D image input or simplified 3D models. Our system incorporates neural networks for tasks that are otherwise complex or computationally expensive to model realistically, such as 3D reconstruction and force calculations, and utilizes a physics-based integration step system to simulate object motion over time using the outputs of these neural networks. We detail the components of the system, evaluate some of our architecture decisions, explore its performance with various input data and network parameters, and propose future extensions.