Distributions for Compositionally Differentiating Parametric Discontinuities
Author(s)
Michel, Jesse
Mu, Kevin
Yang, Xuanda
Bangaru, Sai Praveen
Collins, Elias Rojas
Bernstein, Gilbert
Ragan-Kelley, Jonathan
Carbin, Michael
Li, Tzu-Mao
Date Issued
April 29, 2024
Journal
Proceedings of the ACM on Programming Languages
Publisher
Association for Computing Machinery
Citation
Michel, Jesse, Mu, Kevin, Yang, Xuanda, Bangaru, Sai Praveen, Collins, Elias Rojas et al. 2024. "Distributions for Compositionally Differentiating Parametric Discontinuities." Proceedings of the ACM on Programming Languages, 8 (OOPSLA1).
Version
Final published version
Abstract
Computations in physical simulation, computer graphics, and probabilistic inference often require the differentiation of discontinuous processes due to contact, occlusion, and changes at a point in time. Popular differentiable programming languages, such as PyTorch and JAX, ignore discontinuities during differentiation. This is incorrect for
parametric discontinuities —conditionals containing at least one real-valued parameter and at least one variable of integration. We introduce Potto, the first differentiable first-order programming language to soundly differentiate parametric discontinuities. We present a denotational semantics for programs and program derivatives and show the two accord. We describe the implementation of Potto, which enables separate compilation of programs. Our prototype implementation overcomes previous compile-time bottlenecks achieving an 88.1x and 441.2x speed up in compile time and a 2.5x and 7.9x speed up in runtime, respectively, on two increasingly large image stylization benchmarks. We showcase Potto by implementing a prototype differentiable renderer with separately compiled shaders.
parametric discontinuities —conditionals containing at least one real-valued parameter and at least one variable of integration. We introduce Potto, the first differentiable first-order programming language to soundly differentiate parametric discontinuities. We present a denotational semantics for programs and program derivatives and show the two accord. We describe the implementation of Potto, which enables separate compilation of programs. Our prototype implementation overcomes previous compile-time bottlenecks achieving an 88.1x and 441.2x speed up in compile time and a 2.5x and 7.9x speed up in runtime, respectively, on two increasingly large image stylization benchmarks. We showcase Potto by implementing a prototype differentiable renderer with separately compiled shaders.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-ShareAlike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3649843
