Probabilistic Interpretation of Linear Solvers
Abstract
This manuscript proposes a probabilistic framework for algorithms that iteratively solve unconstrained linear problems $Bx = b$ with positive definite $B$ for $x$. The goal is to replace the point estimates returned by existing methods with a Gaussian posterior belief over the elements of the inverse of $B$, which can be used to estimate errors. Recent probabilistic interpretations of the secant family of quasi-Newton optimization algorithms are extended. Combined with properties of the conjugate gradient algorithm, this leads to uncertainty-calibrated methods with very limited cost overhead over conjugate gradients, a self-contained novel interpretation of the quasi-Newton and conjugate gradient algorithms, and a foundation for new nonlinear optimization methods.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2014
- DOI:
- 10.48550/arXiv.1402.2058
- arXiv:
- arXiv:1402.2058
- Bibcode:
- 2014arXiv1402.2058H
- Keywords:
-
- Mathematics - Optimization and Control;
- Computer Science - Machine Learning;
- Computer Science - Numerical Analysis;
- Mathematics - Numerical Analysis;
- Mathematics - Probability;
- Statistics - Machine Learning;
- 90C53;
- 65F10;
- F.2.1;
- G.1.2;
- G.1.3;
- G.1.6
- E-Print:
- final version, in press at SIAM J Optimization