Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles
Name
5849-Article Text-9074-1-10-20200513.pdf
Description
Published version
Size
1.11 MB
Format
Adobe PDF
Checksum (MD5)
73b6498548b16cf6a0e3497ec7483bab
Author(s)
Jain, Siddhartha
Liu, Ge
Mueller, Jonas
Gifford, David
Date Issued
2020
Journal
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Citation
Jain, Siddhartha, Liu, Ge, Mueller, Jonas and Gifford, David. 2020. "Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles." Proceedings of the AAAI Conference on Artificial Intelligence, 34 (04).
Version
Final published version
Abstract
The inaccuracy of neural network models on inputs that do not stem from the distribution underlying the training data is problematic and at times unrecognized. Uncertainty estimates of model predictions are often based on the variation in predictions produced by a diverse ensemble of models applied to the same input. Here we describe Maximize Overall Diversity (MOD), an approach to improve ensemble-based uncertainty estimates by encouraging larger overall diversity in ensemble predictions across all possible inputs. We apply MOD to regression tasks including 38 Protein-DNA binding datasets, 9 UCI datasets, and the IMDB-Wiki image dataset. We also explore variants that utilize adversarial training techniques and data density estimation. For out-of-distribution test examples, MOD significantly improves predictive performance and uncertainty calibration without sacrificing performance on test data drawn from same distribution as the training data. We also find that in Bayesian optimization tasks, the performance of UCB acquisition is improved via MOD uncertainty estimates.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1609/AAAI.V34I04.5849
