Mathematical analysis of uncertainty in machine learning and deep learning
Name
1341996474-MIT.pdf
Size
7.61 MB
Format
Adobe PDF
Checksum (MD5)
ca554ed63a67ebaa0502c7c5b9479918
Author(s)
Kashimura, Takuya.
Advisor(s)
Bart P.G. Van Parys.
Date Issued
2020
Publisher
Massachusetts Institute of Technology
Abstract
In this paper, we study uncertainty in machine learning and deep learning from the mathematical point of view. Uncertainty is involved in many real-world situations. The Bayesian modelling can handle such uncertainty in machine learning community. However, the traditional deep learning model fails to show uncertainty for its outputs. Recently, at the intersection of the Bayesian modelling and deep learning, a new framework called the Bayesian deep learning (BDL) has been proposed and studied, which enables us to estimate uncertainty of deep learning models. As an example of it, we can review the results of Yarin Gal, in which the famous dropout method can be seen as a Bayesian modelling. We also see that overfitting problem of the framework due to the property of the KL divergence, and review the modified algorithm using o-divergence which generalizes the KL divergence. We also study a confidence band to assess uncertainty of a kernel ridge regression estimator. We propose the formulation to obtain a confidence band as the convex optimization, which enables us to use existing algorithms such as the primal-dual inner point method. The proposed method acquires a more accurate and fast confidence band than a bootstrap algorithm. We also see the effectiveness of our proposed method both in the case of function approximation and an estimate of an actual dataset.
Description
Thesis: S.M. in Engineering and Management, Massachusetts Institute of Technology, Engineering Systems Division, System Design and Management Program, 2020
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 69-72).
Subjects
Engineering Systems Division.
System Design and Management Program.
MIT Department
Massachusetts Institute of Technology. Engineering Systems Division
System Design and Management Program.
Terms of Use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
Persistent DSpace Link
