How to Make a Pizza: Learning a Compositional Layer-Based GAN Model
Name
1906.02839(1).pdf
Size
7.71 MB
Format
Adobe PDF
Checksum (MD5)
be3db978216a739bdbca8ffc2e20a7e4
Author(s)
Papadopoulos, Dim P
Tamaazousti, Youssef
Ofli, Ferda
Weber, Ingmar
Torralba, Antonio
Date Issued
January 2020
Journal
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Papadopoulos, Dim P. et al. "How to Make a Pizza: Learning a Compositional Layer-Based GAN Model." 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019, Long Beach, CA, USA, Institute of Electrical and Electronics Engineers, January 2020. © 2019 IEEE
Version
Author's final manuscript
Abstract
A food recipe is an ordered set of instructions for preparing a particular dish. From a visual perspective, every instruction step can be seen as a way to change the visual appearance of the dish by adding extra objects (e.g., adding an ingredient) or changing the appearance of the existing ones (e.g., cooking the dish). In this paper, we aim to teach a machine how to make a pizza by building a generative model that mirrors this step-by-step procedure. To do so, we learn composable module operations which are able to either add or remove a particular ingredient. Each operator is designed as a Generative Adversarial Network (GAN). Given only weak image-level supervision, the operators are trained to generate a visual layer that needs to be added to or removed from the existing image. The proposed model is able to decompose an image into an ordered sequence of layers by applying sequentially in the right order the corresponding removing modules. Experimental results on synthetic and real pizza images demonstrate that our proposed model is able to: (1) segment pizza toppings in a weakly- supervised fashion, (2) remove them by revealing what is occluded underneath them (i.e., inpainting), and (3) infer the ordering of the toppings without any depth ordering supervision. Code, data, and models are available online.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/cvpr.2019.00819
