Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization
Abstract
We study gradient-based optimization methods obtained by direct Runge-Kutta discretization of the ordinary differential equation (ODE) describing the movement of a heavy-ball under constant friction coefficient. When the function is high order smooth and strongly convex, we show that directly simulating the ODE with known numerical integrators achieve acceleration in a nontrivial neighborhood of the optimal solution. In particular, the neighborhood can grow larger as the condition number of the function increases. Furthermore, our results also hold for nonconvex but quasi-strongly convex objectives. We provide numerical experiments that verify the theoretical rates predicted by our results.
- Publication:
-
arXiv e-prints
- Pub Date:
- May 2019
- DOI:
- 10.48550/arXiv.1905.12436
- arXiv:
- arXiv:1905.12436
- Bibcode:
- 2019arXiv190512436Z
- Keywords:
-
- Mathematics - Optimization and Control
- E-Print:
- arXiv admin note: text overlap with arXiv:1805.00521