Backprop as Functor: A compositional perspective on supervised learning
Name
learners.pdf
Size
214.79 KB
Format
Adobe PDF
Checksum (MD5)
3fdc4106183d9e07efcc8fae5a335d6d
Author(s)
Fong, Brendan C
Spivak, David I
Tuyeras, Remy V
Date Issued
June 2019
Journal
Thirty-Fourth Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
Publisher
Association for Computing Machinery
Citation
Fong, Brendan et al. "Backprop as Functor: A compositional perspective on supervised learning." Thirty-Fourth Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), June 2019, Vancouver, Canada, Association for Computing Machinery, June 2019
Version
Author's final manuscript
Abstract
A supervised learning algorithm searches over a set of functions A→B parametrised by a space P to find the best approximation to some ideal function f:A→B. It does this by taking examples (a,f(a))∈A×B, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent---with respect to a fixed step size and an error function satisfying a certain property---defines a monoidal functor from a category of parametrised functions to this category of update rules. This provides a structural perspective on backpropagation, as well as a broad generalisation of neural networks.
MIT Department
Massachusetts Institute of Technology. Department of Mathematics
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://lics.siglog.org/lics19/
