Randomness and permutations in coordinate descent methods
Name
10107_2019_1438_ReferencePDF.pdf
Size
653.76 KB
Format
Adobe PDF
Checksum (MD5)
502f687382c2b92622605c429bc6165d
Author(s)
Gürbüzbalaban, Mert
Ozdaglar, Asuman
Vanli, Nuri D
Wright, Stephen J
Date Issued
September 30, 2019
Publisher
Springer Berlin Heidelberg
Version
Author's final manuscript
Abstract
Abstract
We consider coordinate descent (CD) methods with exact line search on convex quadratic problems. Our main focus is to study the performance of the CD method that use random permutations in each epoch and compare it to the performance of the CD methods that use deterministic orders and random sampling with replacement. We focus on a class of convex quadratic problems with a diagonally dominant Hessian matrix, for which we show that using random permutations instead of random with-replacement sampling improves the performance of the CD method in the worst-case. Furthermore, we prove that as the Hessian matrix becomes more diagonally dominant, the performance improvement attained by using random permutations increases. We also show that for this problem class, using any fixed deterministic order yields a superior performance than using random permutations. We present detailed theoretical analyses with respect to three different convergence criteria that are used in the literature and support our theoretical results with numerical experiments.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s10107-019-01438-4
