Light source relighting for indoor scene photos with deep neural networks

Abstract

We seek to use deep neural networks to develop a method to detect the light sources in a given image of an indoor scene, computationally adjust their lighting intensity, and re-render the edited scene as an image. By doing so, we can visually relight the image--effectively turning the light source "on" or "off" in the image. This thesis introduces such a method by using Generative Adversarial Networks (GANs) and intervention techniques to this end. The method is composed of a pipeline of processing stages, from detecting the light sources to reconstructing the scene in GAN representation space to performing edits on the GAN representation to fine-grained control over the edited lighting, and we present its results here. The thesis work has a wide range of applications in the field of content creation and image editing.