Triangle Splatting

Abstract

We develop a differentiable rendering method for recovering 3D meshes of scenes from 2D images. Unlike existing approaches, our method does not rely on a differentiable renderers and is compatible with any standard mesh rasterizer. To our knowledge, it is the first mesh-based differentiable rendering method that is not reliant the use of visibility masks entirely. Beyond these conceptual advancements, we implemented a set of highly optimized kernels that enable efficient scene representation on a sparse voxel grid, effectively overcoming the cubic scaling bottleneck faced by similar methods. These innovations result in promising performance on unbounded real-world scenes with complex backgrounds.