A more direct and better variant of New QNewton's method Backtracking for m equations in m variables
Abstract
In this paper we apply the ideas of New QNewton's method directly to a system of equations, utilising the specialties of the cost function $f=F^2$, where $F=(f_1,\ldots ,f_m)$. The first algorithm proposed here is a modification of LevenbergMarquardt algorithm, where we prove some new results on global convergence and avoidance of saddle points. The second algorithm proposed here is a modification of New QNewton's method Backtracking, where we use the operator $\nabla ^2f(x)+\delta F(x)^{\tau}$ instead of $\nabla ^2f(x)+\delta \nabla f(x)^{\tau}$. This new version is more suitable than New QNewton's method Backtracking itself, while currently has better avoidance of saddle points guarantee than LevenbergMarquardt algorithms. Also, a general scheme for second order methods for solving systems of equations is proposed. We will also discuss a way to avoid that the limit of the constructed sequence is a solution of $H(x)^{\intercal}F(x)=0$ but not of $F(x)=0$.
 Publication:

arXiv eprints
 Pub Date:
 October 2021
 arXiv:
 arXiv:2110.07403
 Bibcode:
 2021arXiv211007403T
 Keywords:

 Mathematics  Algebraic Geometry;
 Mathematics  Complex Variables;
 Mathematics  Dynamical Systems;
 Mathematics  Numerical Analysis;
 Mathematics  Optimization and Control
 EPrint:
 11 pages. New algorithms and results (including avoidance of saddle points) are added. Relevant references are added. Some typos fixed