2026-09-18T23:24:48Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1269192021-07-05T14:03:20Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Kamal Youcef-Toumi.
Tohme, Tony.
Massachusetts Institute of Technology. Computation for Design and Optimization Program.
Massachusetts Institute of Technology. Computation for Design and Optimization Program
2020-09-03T16:40:27Z
2020-09-03T16:40:27Z
2020
2020
https://hdl.handle.net/1721.1/126919
1191254179
Thesis: S.M., Massachusetts Institute of Technology, Computation for Design and Optimization Program, May, 2020
Cataloged from the official PDF of thesis.
Includes bibliographical references (pages 109-114).
In model development, model calibration and validation play complementary roles toward learning reliable models. In this thesis, we propose and develop the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We show that the BVM can represent all the standard validation metrics - square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing - as special cases while improving, generalizing and further quantifying their uncertainties. In addition, the BVM assists users and analysts in designing and selecting their models by allowing them to specify their own validation conditions and requirements. Further, we expand the BVM framework to a general calibration and validation framework by inverting the validation mathematics into a method for generalized Bayesian regression and model learning. We perform Bayesian regression based on a user's definition of model-data agreement. This allows for model selection on any type of data distribution, unlike Bayesian and standard regression techniques, that "fail" in some cases. We show that our tool is capable of representing and combining Bayesian regression, standard regression, and likelihood-based calibration techniques in a single framework while being able to generalize aspects of these methods. This tool also offers new insights into the interpretation of the predictive envelopes in Bayesian regression, standard regression, and likelihood-based methods while giving the analyst more control over these envelopes.
by Tony Tohme.
S.M.
S.M. Massachusetts Institute of Technology, Computation for Design and Optimization Program
114 pages
eng
Massachusetts Institute of Technology
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
http://dspace.mit.edu/handle/1721.1/7582
Computation for Design and Optimization Program.
The Bayesian validation metric : a framework for probabilistic model calibration and validation
Thesis
application/pdf
2020-09-03T16:40:27Z
Publication
Master
CDO
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="929ca7b3-e4c9-4c00-8908-4f1e0decd719">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>The Bayesian validation metric : a framework for probabilistic model calibration and validation</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2020</PublicationDate>
<Authors>
<Author>
<DisplayName>Tohme, Tony.</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Computation for Design and Optimization Program.</Keyword>
<Abstract>In model development, model calibration and validation play complementary roles toward learning reliable models. In this thesis, we propose and develop the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We show that the BVM can represent all the standard validation metrics - square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing - as special cases while improving, generalizing and further quantifying their uncertainties. In addition, the BVM assists users and analysts in designing and selecting their models by allowing them to specify their own validation conditions and requirements. Further, we expand the BVM framework to a general calibration and validation framework by inverting the validation mathematics into a method for generalized Bayesian regression and model learning. We perform Bayesian regression based on a user's definition of model-data agreement. This allows for model selection on any type of data distribution, unlike Bayesian and standard regression techniques, that "fail" in some cases. We show that our tool is capable of representing and combining Bayesian regression, standard regression, and likelihood-based calibration techniques in a single framework while being able to generalize aspects of these methods. This tool also offers new insights into the interpretation of the predictive envelopes in Bayesian regression, standard regression, and likelihood-based methods while giving the analyst more control over these envelopes.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>