Dynamic behavior for a gradient algorithm with energy and momentum
Abstract
This paper investigates a novel gradient algorithm, AGEM, using both energy and momentum, for addressing general non-convex optimization problems. The solution properties of the AGEM algorithm, including aspects such as uniformly boundedness and convergence to critical points, are examined. The dynamic behavior is studied through a comprehensive analysis of a high-resolution ODE system. This ODE system, being nonlinear, is derived by taking the limit of the discrete scheme while preserving the momentum effect through a rescaling of the momentum parameter. The paper emphasizes the global well-posedness of the ODE system and the time-asymptotic convergence of solution trajectories. Furthermore, we establish a linear convergence rate for objective functions that adhere to the Polyak-Łojasiewicz condition.
- Publication:
-
arXiv e-prints
- Pub Date:
- March 2022
- DOI:
- 10.48550/arXiv.2203.12199
- arXiv:
- arXiv:2203.12199
- Bibcode:
- 2022arXiv220312199L
- Keywords:
-
- Mathematics - Optimization and Control;
- Mathematics - Classical Analysis and ODEs;
- 65K10 (Primary) 90C15 (Secondary)
- E-Print:
- 30 pages, 2 figures