Sufficient Conditions for Uniform Stability of Regularization Algorithms
Author(s)
Poggio, Tomaso; Rosasco, Lorenzo; Wibisono, Andre
DownloadMIT-CSAIL-TR-2009-060.pdf (284.7Kb)
Other Contributors
Center for Biological and Computational Learning (CBCL)
Advisor
Tomaso Poggio
Metadata
Show full item recordAbstract
In this paper, we study the stability and generalization properties of penalized empirical-risk minimization algorithms. We propose a set of properties of the penalty term that is sufficient to ensure uniform ?-stability: we show that if the penalty function satisfies a suitable convexity property, then the induced regularization algorithm is uniformly ?-stable. In particular, our results imply that regularization algorithms with penalty functions which are strongly convex on bounded domains are ?-stable. In view of the results in [3], uniform stability implies generalization, and moreover, consistency results can be easily obtained. We apply our results to show that â p regularization for 1 < p <= 2 and elastic-net regularization are uniformly ?-stable, and therefore generalize.
Date issued
2009-12-01Series/Report no.
CBCL-284MIT-CSAIL-TR-2009-060
Keywords
artificial intelligence, theory, computation, learning