On matrix factorization and efficient least squares solution.
Abstract
Least squares solution of ill conditioned normal equations by CholeskyBanachiewicz (ChB) factorization suffers from numerical problems related to near singularity and loss of accuracy. We demonstrate that the near singularity does not arise for correctly posed statistical problems. The accuracy loss is also immaterial since for nonlinear least squares the solution by Newton Raphson iterations yields machine accuracy with no regard for accuracy of an individual iteration (Wilkinson 1963). Since this accuracy may not be achieved using singular value decomposition (SVD) without additional iterations for differential corrections and since SVD is more demanding in terms of number of operations and particularly in terms of required memory, we argue that ChB factorization remains the algorithm of choice for least squares. We present a new, very compact implementation in code of Cholesky (1924) and Banachiewicz (1938b) factorization in an elegant form proposed by Banachiewicz (1942). Source listing of the code is provided. We point out that in the same publication Banachiewicz (1938) discovered LU factorization of square matrices before Crout (1941) and rediscovered factorization of the symmetric matrices after Cholesky (1924). Since the two algorithms became confused, no due credit is given to Banachiewicz in modern literature.
 Publication:

Astronomy and Astrophysics Supplement Series
 Pub Date:
 April 1995
 Bibcode:
 1995A&AS..110..405S
 Keywords:

 METHODS: NUMERICAL;
 METHODS: DATA ANALYSIS