MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms for Sparse Signal Recovery in the Presence of Noise

Author(s)
Babadi, Behtash; Brown, Emery N.; Ba, Demba E.; Purdon, Patrick Lee
Thumbnail
DownloadBrown_Convergence and stability.pdf (318.6Kb)
OPEN_ACCESS_POLICY

Open Access Policy

Creative Commons Attribution-Noncommercial-Share Alike

Alternative title
Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms
Terms of use
Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/
Metadata
Show full item record
Abstract
In this paper, we study the theoretical properties of iteratively re-weighted least squares (IRLS) algorithms and their utility in sparse signal recovery in the presence of noise. We demonstrate a one-to-one correspondence between the IRLS algorithms and a class of Expectation-Maximization (EM) algorithms for constrained maximum likelihood estimation under a Gaussian scale mixture (GSM) distribution. The EM formalism, as well as the connection to GSMs, allow us to establish that the IRLS algorithms minimize smooth versions of the lν `norms', for . We leverage EM theory to show that the limit points of the sequence of IRLS iterates are stationary points of the smooth lν “norm” minimization problem on the constraint set. We employ techniques from Compressive Sampling (CS) theory to show that the IRLS algorithm is stable, if the limit point of the iterates coincides with the global minimizer. We further characterize the convergence rate of the IRLS algorithm, which implies global linear convergence for ν = 1 and local super-linear convergence for . We demonstrate our results via simulation experiments. The simplicity of IRLS, along with the theoretical guarantees provided in this contribution, make a compelling case for its adoption as a standard tool for sparse signal recovery.
Date issued
2013-10
URI
http://hdl.handle.net/1721.1/86328
Department
Harvard University--MIT Division of Health Sciences and Technology; Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Journal
IEEE Transactions on Signal Processing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Ba, Demba, Behtash Babadi, Patrick L. Purdon, and Emery N. Brown. “Convergence and Stability of Iteratively Re-Weighted Least Squares Algorithms.” IEEE Transactions on Signal Processing 62, no. 1 (n.d.): 183–195.
Version: Original manuscript
ISSN
1053-587X
1941-0476

Collections
  • MIT Open Access Articles

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.