Towards Understanding Generalization via Analytical Learning Theory
Name
MIT-CSAIL-TR-2018-019.pdf
Size
383.33 KB
Format
Adobe PDF
Checksum (MD5)
b3208c4463795f56886f74700dd60fbe
Author(s)
Kawaguchi, Kenji
Benigo, Yoshua
Verma, Vikas
Kaelbling, Leslie Pack
Date Issued
October 1, 2018
Series/Report no.
;MIT-CSAIL-TR-2018-019
Abstract
This paper introduces a novel measure-theoretic theory for machine learning
that does not require statistical assumptions. Based on this theory, a new
regularization method in deep learning is derived and shown to outperform
previous methods in CIFAR-10, CIFAR-100, and SVHN. Moreover, the proposed
theory provides a theoretical basis for a family of practically successful
regularization methods in deep learning. We discuss several consequences of
our results on one-shot learning, representation learning, deep learning,
and curriculum learning. Unlike statistical learning theory, the proposed
learning theory analyzes each problem instance individually via measure
theory, rather than a set of problem instances via statistics. As a result,
it provides different types of results and insights when compared to
statistical learning theory.
Subjects
Machine Learning
Measure Theory
Regularization method
Neural Network
Persistent DSpace Link
