The Bayesian validation metric : a framework for probabilistic model calibration and validation
Name
1191254179-MIT.pdf
Size
2.51 MB
Format
Adobe PDF
Checksum (MD5)
7ecb18ad6703f2fa3d8ede0c8c88d9db
Author(s)
Tohme, Tony.
Advisor(s)
Kamal Youcef-Toumi.
Date Issued
2020
Publisher
Massachusetts Institute of Technology
Abstract
In model development, model calibration and validation play complementary roles toward learning reliable models. In this thesis, we propose and develop the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We show that the BVM can represent all the standard validation metrics - square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing - as special cases while improving, generalizing and further quantifying their uncertainties. In addition, the BVM assists users and analysts in designing and selecting their models by allowing them to specify their own validation conditions and requirements. Further, we expand the BVM framework to a general calibration and validation framework by inverting the validation mathematics into a method for generalized Bayesian regression and model learning. We perform Bayesian regression based on a user's definition of model-data agreement. This allows for model selection on any type of data distribution, unlike Bayesian and standard regression techniques, that "fail" in some cases. We show that our tool is capable of representing and combining Bayesian regression, standard regression, and likelihood-based calibration techniques in a single framework while being able to generalize aspects of these methods. This tool also offers new insights into the interpretation of the predictive envelopes in Bayesian regression, standard regression, and likelihood-based methods while giving the analyst more control over these envelopes.
Description
Thesis: S.M., Massachusetts Institute of Technology, Computation for Design and Optimization Program, May, 2020
Cataloged from the official PDF of thesis.
Includes bibliographical references (pages 109-114).
Subjects
Computation for Design and Optimization Program.
MIT Department
Massachusetts Institute of Technology. Computation for Design and Optimization Program
Terms of Use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
Persistent DSpace Link
