Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees
Name
Tight bounds.pdf
Size
332.1 KB
Format
Adobe PDF
Checksum (MD5)
7eeab154d5a2bfa6a5b187850e485566
Author(s)
Honorio Carrillo, Jean
Jaakkola, Tommi S.
Date Issued
April 2014
Journal
Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS)
Publisher
Journal of Machine Learning Research
Citation
Honorio, Jean and Tomi Jaakkola. "Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees." Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS), 22-25 April 2014, Reykjavik, Iceland, Journal of Machine Learning Research, 2014.
Version
Author's final manuscript
Abstract
We analyze the expected risk of linear classifiers for a fixed weight vector in the “minimax” setting. That is, we analyze the worst-case risk among all data distributions with a given mean and covariance. We provide a simpler proof of the tight polynomial-tail bound for general random variables. For sub-Gaussian random variables, we derive a novel tight exponential bound. We also provide new PAC-Bayes finite-sample guarantees when training data
is available. Our “minimax” generalization bounds are dimensionality-independent and O(√1/m) for m samples.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
http://proceedings.mlr.press/v33/
