Optimized first-order methods for smooth convex minimization
Abstract
We introduce new optimized first-order methods for smooth unconstrained convex minimization. Drori and Teboulle recently described a numerical method for computing the $N$-iteration optimal step coefficients in a class of first-order algorithms that includes gradient methods, heavy-ball methods, and Nesterov's fast gradient methods. However, Drori and Teboulle's numerical method is computationally expensive for large $N$, and the corresponding numerically optimized first-order algorithm requires impractical memory and computation for large-scale optimization problems. In this paper, we propose optimized first-order algorithms that achieve a convergence bound that is two times smaller than for Nesterov's fast gradient methods; our bound is found analytically and refines the numerical bound. Furthermore, the proposed optimized first-order methods have efficient recursive forms that are remarkably similar to Nesterov's fast gradient methods.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2014
- DOI:
- 10.48550/arXiv.1406.5468
- arXiv:
- arXiv:1406.5468
- Bibcode:
- 2014arXiv1406.5468K
- Keywords:
-
- Mathematics - Optimization and Control
- E-Print:
- Math. Prog. 159(1):81-107, Sep. 2016