Nonparametric Sparsity and Regularization
Name
MIT-CSAIL-TR-2011-041.pdf
Size
663.99 KB
Format
Adobe PDF
Checksum (MD5)
36d88542803d0394fec83dc1f7d23ab9
Author(s)
Mosci, Sofia
Rosasco, Lorenzo
Santoro, Matteo
Verri, Alessandro
Villa, Silvia
Advisor(s)
Tomaso Poggio
Date Issued
September 26, 2011
Series/Report no.
MIT-CSAIL-TR-2011-041
CBCL-303
Abstract
In this work we are interested in the problems of supervised learning and variable selection when the input-output dependence is described by a nonlinear function depending on a few variables. Our goal is to consider a sparse nonparametric model, hence avoiding linear or additive models. The key idea is to measure the importance of each variable in the model by making use of partial derivatives. Based on this intuition we propose and study a new regularizer and a corresponding least squares regularization scheme. Using concepts and results from the theory of reproducing kernel Hilbert spaces and proximal methods, we show that the proposed learning algorithm corresponds to a minimization problem which can be provably solved by an iterative procedure. The consistency properties of the obtained estimator are studied both in terms of prediction and selection performance. An extensive empirical analysis shows that the proposed method performs favorably with respect to the state-of-the-art.
Subjects
computational learning
machine learning
Persistent DSpace Link
