Iterative Projection Methods for Structured Sparsity Regularization
Name
MIT-CSAIL-TR-2009-050.pdf
Size
510.37 KB
Format
Adobe PDF
Checksum (MD5)
37a865ebb946bf789d6b24eb42f9230d
Author(s)
Rosasco, Lorenzo
Verri, Alessandro
Santoro, Matteo
Mosci, Sofia
Villa, Silvia
Advisor(s)
Tomaso Poggio
Date Issued
October 14, 2009
Series/Report no.
MIT-CSAIL-TR-2009-050
CBCL-282
Abstract
In this paper we propose a general framework to characterize and solve the optimization problems underlying a large class of sparsity based regularization algorithms. More precisely, we study the minimization of learning functionals that are sums of a differentiable data term and a convex non differentiable penalty. These latter penalties have recently become popular in machine learning since they allow to enforce various kinds of sparsity properties in the solution. Leveraging on the theory of Fenchel duality and subdifferential calculus, we derive explicit optimality conditions for the regularized solution and propose a general iterative projection algorithm whose convergence to the optimal solution can be proved. The generality of the framework is illustrated, considering several examples of regularization schemes, including l1 regularization (and several variants), multiple kernel learning and multi-task learning. Finally, some features of the proposed framework are empirically studied.
Subjects
computation
learning
Persistent DSpace Link
