A more direct and better variant of New Q-Newton's method Backtracking for m equations in m variables
Abstract
In this paper we apply the ideas of New Q-Newton's method directly to a system of equations, utilising the specialties of the cost function $f=||F||^2$, where $F=(f_1,\ldots ,f_m)$. The first algorithm proposed here is a modification of Levenberg-Marquardt algorithm, where we prove some new results on global convergence and avoidance of saddle points. The second algorithm proposed here is a modification of New Q-Newton's method Backtracking, where we use the operator $\nabla ^2f(x)+\delta ||F(x)||^{\tau}$ instead of $\nabla ^2f(x)+\delta ||\nabla f(x)||^{\tau}$. This new version is more suitable than New Q-Newton's method Backtracking itself, while currently has better avoidance of saddle points guarantee than Levenberg-Marquardt algorithms. Also, a general scheme for second order methods for solving systems of equations is proposed. We will also discuss a way to avoid that the limit of the constructed sequence is a solution of $H(x)^{\intercal}F(x)=0$ but not of $F(x)=0$.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2021
- DOI:
- 10.48550/arXiv.2110.07403
- arXiv:
- arXiv:2110.07403
- Bibcode:
- 2021arXiv211007403T
- Keywords:
-
- Mathematics - Algebraic Geometry;
- Mathematics - Complex Variables;
- Mathematics - Dynamical Systems;
- Mathematics - Numerical Analysis;
- Mathematics - Optimization and Control
- E-Print:
- 11 pages. New algorithms and results (including avoidance of saddle points) are added. Relevant references are added. Some typos fixed