Differentiable programming for image processing and deep learning in halide
Name
li_durand.pdf
Size
6.36 MB
Format
Adobe PDF
Checksum (MD5)
433f0e9b0fbf3a40b376591c3a7accc2
Author(s)
Li, Tzu-Mao
Gharbi, Michael Yanis
Adams, Andrew
Durand, Frederic
Ragan-Kelley, Jonathan
Date Issued
August 2018
Journal
ACM Transactions on Graphics
Publisher
Association for Computing Machinery (ACM)
Citation
Li, Tzu-Mao et al. "Differentiable programming for image processing and deep learning in halide." ACM Transactions on Graphics 37, 4 (August 2018): 139 © 2018 The Authors
Version
Author's final manuscript
Abstract
Gradient-based optimization has enabled dramatic advances in computational imaging through techniques like deep learning and nonlinear optimization. These methods require gradients not just of simple mathematical functions, but of general programs which encode complex transformations of images and graphical data. Unfortunately, practitioners have traditionally been limited to either hand-deriving gradients of complex computations, or composing programs from a limited set of coarse-grained operators in deep learning frameworks. At the same time, writing programs with the level of performance needed for imaging and deep learning is prohibitively difficult for most programmers.
We extend the image processing language Halide with general reverse-mode automatic differentiation (AD), and the ability to automatically optimize the implementation of gradient computations. This enables automatic computation of the gradients of arbitrary Halide programs, at high performance, with little programmer effort. A key challenge is to structure the gradient code to retain parallelism. We define a simple algorithm to automatically schedule these pipelines, and show how Halide's existing scheduling primitives can express and extend the key AD optimization of "checkpointing."
Using this new tool, we show how to easily define new neural network layers which automatically compile to high-performance GPU implementations, and how to solve nonlinear inverse problems from computational imaging. Finally, we show how differentiable programming enables dramatically improving the quality of even traditional, feed-forward image processing algorithms, blurring the distinction between classical and deep methods.
MIT Department
Massachusetts Institute of Technology. Laboratory for Computer Science
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3197517.3201383
