From bits to information : learning meets compressive sensing

Abstract

A quantization approach to supervised learning, compressive sensing, and phase retrieval is presented in this thesis. We introduce a set of common techniques that allow us, in those three settings, to represent high dimensional data using the order statistics of linear and non linear measurements. We introduce new algorithms for signals classification in the multiclass and the multimodal settings, as well as algorithms for signals representation and recovery from quantized linear and quadratic measurements. We analyze the statistical consistency of our algorithms and prove their robustness to different sources of perturbation, as well as their computational efficiency. We present and analyze applications of our theoretical results in realistic setups, such as computer vision classification tasks, Audio-Visual Automatic Speech Recognition, lossy image compression and retrieval via locality sensitive hashing, locally linear estimation in large scale learning and Fourier sampling for phase retrieval - of particular interest in X-ray crystallography and super-resolution diffraction imaging applications. Our analysis of quantization based algorithms highlights interesting tradeoffs between memory complexity, sample complexity, and time complexity in algorithms design.