Overcomplete independent component analysis via SDP
Name
podosinnikova19a.pdf
Description
Published version
Size
1.15 MB
Format
Adobe PDF
Checksum (MD5)
c838756983422aca73360b0f14481189
Author(s)
Podosinnikova, Anastasia
Perry, Amelia E.
Wein, Alexander Spence
Sontag, David Alexander
Date Issued
April 2019
Journal
Proceedings of Machine Learning Research
Publisher
International Machine Learning Society
Citation
Podosinnikova, Anastasia et al. “Overcomplete independent component analysis via SDP.” Paper in the Proceedings of Machine Learning Research, 89, 22nd International Conference on Artificial Intelligence and Statistics (AISTATS) 2019, Naha,Okinawa, Japan, April 16-18 2019, International Machine Learning Society © 2019 The Author(s)
Version
Final published version
Abstract
We present a novel algorithm for overcomplete independent components analysis (ICA), where the number of latent sources k exceeds the dimension p of observed variables. Previous algorithms either suffer from high computational complexity or make strong assumptions about the form of the mixing matrix. Our algorithm does not make any sparsity assumption yet enjoys favorable computational and theoretical properties. Our algorithm consists of two main steps: (a) estimation of the Hessians of the cumulant generating function (as opposed to the fourth and higher order cumulants used by most algorithms) and (b) a novel semi-definite programming (SDP) relaxation for recovering a mixing component. We show that this relaxation can be efficiently solved with a projected accelerated gradient descent method, which makes the whole algorithm computationally practical. Moreover, we conjecture that the proposed program recovers a mixing component at the rate k < p2/4 and prove that a mixing component can be recovered with high probability when k < (2 - ")plog p when the original components are sampled uniformly at random on the hypersphere. Experiments are provided on synthetic data and the CIFAR-10 dataset of real images.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://proceedings.mlr.press/v89/
