Matrix probing, skeleton decompositions, and sparse Fourier transform

Abstract

In this thesis, we present three different randomized algorithms that help to solve matrices, compute low rank approximations and perform the Fast Fourier Transform. Matrix probing and its conditioning When a matrix A with n columns is known to be well approximated by a linear combination of basis matrices B1,... , Bp, we can apply A to a random vector and solve a linear system to recover this linear combination. The same technique can be used to obtain an approximation to A-1. A basic question is whether this linear system is well-conditioned. This is important for two reasons: a well-conditioned system means (1) we can invert it and (2) the error in the reconstruction can be controlled. In this paper, we show that if the Gram matrix of the Bj's is sufficiently well-conditioned and each Bj has a high numerical rank, then n [alpha] p log2 n will ensure that the linear system is well-conditioned with high probability. Our main application is probing linear operators with smooth pseudodifferential symbols such as the wave equation Hessian in seismic imaging. We also demonstrate numerically that matrix probing can produce good preconditioners for inverting elliptic operators in variable media. Skeleton decompositions in sublinear time A skeleton decomposition of a matrix A is any factorization of the form A:CZAR: where A:C comprises columns of A, and AR: comprises rows of A. In this paper, we investigate the conditions under which random sampling of C and R results in accurate skeleton decompositions. When the singular vectors (or more generally the generating vectors) are incoherent, we show that a simple algorithm returns an accurate skeleton in sublinear O(l3) time from l ~/- k logn rows and columns drawn uniformly at random, with an approximation error of the form O(n/l[sigma]k) where 0k is the k-th singular value of A. We discuss the crucial role that regularization plays in forming the middle matrix U as a pseudo-inverse of the restriction ARC of A to rows in R and columns in C. The proof methods enable the analysis of two alternative sublinear-time algorithms, based on the rank-revealing QR decomposition, which allow us to tighten the number of rows and/or columns sampled to k with an error bound proportional to [sigma]-k. Sparse Fourier transform using the matrix pencil method One of the major applications of the FFT is to compress frequency-sparse signals. Yet, FFT algorithms do not leverage on this sparsity. Say we want to perform the Fourier transform on [epsilon] E CN to obtain some [chi], which is known to be S-sparse with some additive noise. Even when S is small, FFT still takes O(N log N) time. In contrast, SFT (sparse Fourier transform) algorithms aim to run in Õ(S)-time ignoring log factors. Unfortunately, SFT algorithms are not widely used because they are faster than the FFT only when S << N. We hope to address this deficiency. In this work, we present the fastest known robust Õ(S)-time algorithm which can run up to 20 times faster than the current state-of-the-art algorithm AAFFT. The major new ingredient is a mode collision detector using the matrix pencil method. This enables us to do away with a time-consuming coefficient estimation loop, use a cheaper filter and take fewer samples of x. We also speed up a crucial basic operation of many SFT algorithms by halving the number of trigonometric computations. Our theory is however not complete. First, we prove that the collision detector works for a few classes of random signals. Second, we idealize the behavior of the collision detector and show that with good probability, our algorithm runs in O(S log 2 - log N) time and outputs a O(S)-sparse [chi]' such that [mathematical formula inserted] where [chi], is the best exact S-sparse approximation of [chi].