Direct optimization through arg max for discrete variational auto-encoder
Name
NeurIPS-2019-direct-optimization-through-arg-max-for-discrete-variational-auto-encoder-Paper.pdf
Description
Published version
Size
2.45 MB
Format
Adobe PDF
Checksum (MD5)
0e795fcfbf4262b80a6df41f63787d1b
Author(s)
Gane, Andreea
Jaakkola, Tommi S
Date Issued
December 2019
Journal
33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Publisher
Morgan Kaufmann Publishers
Citation
Lorberbom, Guy et al. “Direct optimization through arg max for discrete variational auto-encoder.” 33rd Conference on Neural Information Processing Systems, December 2019, Vancouver, Canada, Morgan Kaufmann Publishers, 2019. © 2019 The Author(s)
Version
Final published version
Abstract
Reparameterization of variational auto-encoders with continuous random variables is an effective method for reducing the variance of their gradient estimates. In the discrete case, one can perform reparametrization using the Gumbel-Max trick, but the resulting objective relies on an arg max operation and is non-differentiable. In contrast to previous works which resort to softmax-based relaxations, we propose to optimize it directly by applying the direct loss minimization approach. Our proposal extends naturally to structured discrete latent variable models when evaluating the arg max operation is tractable. We demonstrate empirically the effectiveness of the direct loss minimization technique in variational autoencoders with both unstructured and structured discrete latent variables.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
