On the Complexity Analysis of Randomized BlockCoordinate Descent Methods
Abstract
In this paper we analyze the randomized blockcoordinate descent (RBCD) methods proposed in [8,11] for minimizing the sum of a smooth convex function and a blockseparable convex function. In particular, we extend Nesterov's technique developed in [8] for analyzing the RBCD method for minimizing a smooth convex function over a blockseparable closed convex set to the aforementioned more general problem and obtain a sharper expectedvalue type of convergence rate than the one implied in [11]. Also, we obtain a better highprobability type of iteration complexity, which improves upon the one in [11] by at least the amount $O(n/\epsilon)$, where $\epsilon$ is the target solution accuracy and $n$ is the number of problem blocks. In addition, for unconstrained smooth convex minimization, we develop a new technique called {\it randomized estimate sequence} to analyze the accelerated RBCD method proposed by Nesterov [11] and establish a sharper expectedvalue type of convergence rate than the one given in [11].
 Publication:

arXiv eprints
 Pub Date:
 May 2013
 arXiv:
 arXiv:1305.4723
 Bibcode:
 2013arXiv1305.4723L
 Keywords:

 Mathematics  Optimization and Control;
 Computer Science  Machine Learning;
 Computer Science  Numerical Analysis;
 Mathematics  Numerical Analysis;
 Statistics  Machine Learning
 EPrint:
 26 pages (submitted)