Achieving Acceleration in Distributed Optimization via Direct Discretization of the Heavy-Ball ODE
Abstract
We develop a distributed algorithm for convex Empirical Risk Minimization, the problem of minimizing large but finite sum of convex functions over networks. The proposed algorithm is derived from directly discretizing the second-order heavy-ball differential equation and results in an accelerated convergence rate, i.e, faster than distributed gradient descent-based methods for strongly convex objectives that may not be smooth. Notably, we achieve acceleration without resorting to the well-known Nesterov's momentum approach. We provide numerical experiments and contrast the proposed method with recently proposed optimal distributed optimization algorithms.
- Publication:
-
arXiv e-prints
- Pub Date:
- November 2018
- DOI:
- 10.48550/arXiv.1811.02521
- arXiv:
- arXiv:1811.02521
- Bibcode:
- 2018arXiv181102521Z
- Keywords:
-
- Mathematics - Optimization and Control