Mesh regularization for multi-view shape reconstruction via inverse graphics
Name
1251779679-MIT.pdf
Size
13.25 MB
Format
Adobe PDF
Checksum (MD5)
e72f55e3c37b91a53844d2668c51182c
Author(s)
Ganeshan, Sanjay.
Advisor(s)
Frédo Durand.
Date Issued
2021
Publisher
Massachusetts Institute of Technology
Abstract
Inverse rendering uses 2D images to infer the 3D scene parameters that produced the images. Unfortunately, inverse rendering is an ill-posed problem that is difficult to optimize. In this paper, we explore using a differentiable renderer to solve inverse rendering problems that are constrained to a single object with two unknowns: shape and texture. The inferred 3D scene is thus a single 3D model. We iteratively optimize an input "guess" 3D model to fit a set of input target images of the desired 3D object. The constrained inverse rendering problem remains ill-posed. So, we adapt, develop, and evaluate a number of deformation, regularization, and training techniques that allow the optimization to converge to visually appealing output 3D models. The system consumes a 3D triangle mesh and target images as input. It outputs a 3D mesh and its corresponding 2D texture image that can easily be consumed by other programs. The best results are achieved using a deep-mesh prior neural network, an Image Pyramid coarse-to-fine loss function, a Silhouette Loss function that is robust to inaccuracies in texture, three separate mesh regularizing loss functions - Laplacian, Flatness, and Non-Uniformity, and periodic refinement operations where the output mesh is upsampled and its manifold is recomputed.
Description
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, February, 2021
Cataloged from the official PDF of thesis.
Includes bibliographical references (pages 115-117).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
Persistent DSpace Link
