A unified framework for consistency of regularized loss minimizers
Name
Jaakkola_A unified.pdf
Size
384.6 KB
Format
Adobe PDF
Checksum (MD5)
dcb67b68836dd448afebfde3700af22b
Author(s)
Honorio, Jean
Jaakkola, Tommi S.
Date Issued
2014
Journal
Journal of Machine Learning Research: Workshop and Conference Proceedings
Publisher
Association for Computing Machinery (ACM)
Citation
Honorio, Jean, and Tommi Jaakkola. "A unified framework for consistency of regularized loss minimizers." Journal of Machine Learning Research: Workshop and Conference Proceedings, Proceedings of The 31st International Conference on Machine Learning, Volume 32 (2014), 136-144.
Version
Final published version
Abstract
We characterize a family of regularized loss minimization problems that satisfy three properties: scaled uniform convergence, super-norm regularization, and norm-loss monotonicity. We show several theoretical guarantees within this framework, including loss consistency, norm consistency, sparsistency (i.e. support recovery) as well as sign consistency. A number of regularization problems can be shown to fall within our framework and we provide several examples. Our results can be seen as a concise summary of existing guarantees but we also extend them to new settings. Our formulation enables us to assume very little about the hypothesis class, data distribution, the loss, or the regularization. In particular, many of our results do not require a bounded hypothesis class, or identically distributed samples. Similarly, we do not assume boundedness, convexity or smoothness of the loss nor the regularizer. We only assume approximate optimality of the empirical minimizer. In terms of recovery, in contrast to existing results, our sparsistency and sign consistency results do not require knowledge of the sub-differential of the objective function.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://jmlr.org/proceedings/papers/v32/
