A unified framework for consistency of regularized loss minimizers
Author(s)
Honorio, Jean; Jaakkola, Tommi S.
DownloadJaakkola_A unified.pdf (384.5Kb)
PUBLISHER_POLICY
Publisher Policy
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Terms of use
Metadata
Show full item recordAbstract
We characterize a family of regularized loss minimization problems that satisfy three properties: scaled uniform convergence, super-norm regularization, and norm-loss monotonicity. We show several theoretical guarantees within this framework, including loss consistency, norm consistency, sparsistency (i.e. support recovery) as well as sign consistency. A number of regularization problems can be shown to fall within our framework and we provide several examples. Our results can be seen as a concise summary of existing guarantees but we also extend them to new settings. Our formulation enables us to assume very little about the hypothesis class, data distribution, the loss, or the regularization. In particular, many of our results do not require a bounded hypothesis class, or identically distributed samples. Similarly, we do not assume boundedness, convexity or smoothness of the loss nor the regularizer. We only assume approximate optimality of the empirical minimizer. In terms of recovery, in contrast to existing results, our sparsistency and sign consistency results do not require knowledge of the sub-differential of the objective function.
Date issued
2014Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer ScienceJournal
Journal of Machine Learning Research: Workshop and Conference Proceedings
Publisher
Association for Computing Machinery (ACM)
Citation
Honorio, Jean, and Tommi Jaakkola. "A unified framework for consistency of regularized loss minimizers." Journal of Machine Learning Research: Workshop and Conference Proceedings, Proceedings of The 31st International Conference on Machine Learning, Volume 32 (2014), 136-144.
Version: Final published version
ISSN
1938-7228