Elastic-Net Regularization in Learning Theory
Citable URI:
http://hdl.handle.net/1721.1/41889
| Title: | Elastic-Net Regularization in Learning Theory |
| Author: | De Mol, Christine; Rosasco, Lorenzo; De Vito, Ernesto |
| Other Contributors: | Center for Biological and Computational Learning (CBCL) |
| Advisor: | Tomaso Poggio |
| Issue Date: | 2008-07-24 |
| Abstract: | Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie ["Regularization and variable selection via the elastic net" J. R. Stat. Soc. Ser. B, 67(2):301-320, 2005] for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combination of elements (features) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular "elastic-net representation" of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed in "Regularization and variable selection via the elastic net". |
| URI: | http://hdl.handle.net/1721.1/41889 |
| Series/Report no.: | MIT-CSAIL-TR-2008-046, CBCL-273 |
| Keywords: | machine learning, regularization, feature selection |
Files in this item
| Files | Size | Format |
|---|---|---|
| MIT-CSAIL-TR-2008-046.pdf | 462.2Kb | application/pdf |
| MIT-CSAIL-TR-2008-046.ps | 73.87Kb | application/postscript |
The following license files are associated with this item:
This item appears in the following Collection(s)
All Items in DSpace@MIT are protected by original copyright, with all rights reserved, unless otherwise indicated.
|
|
Contact
MIT Libraries
|