MessagePassing Algorithms for Quadratic Minimization
Abstract
Gaussian belief propagation (GaBP) is an iterative algorithm for computing the mean of a multivariate Gaussian distribution, or equivalently, the minimum of a multivariate positive definite quadratic function. Sufficient conditions, such as walksummability, that guarantee the convergence and correctness of GaBP are known, but GaBP may fail to converge to the correct solution given an arbitrary positive definite quadratic function. As was observed in previous work, the GaBP algorithm fails to converge if the computation trees produced by the algorithm are not positive definite. In this work, we will show that the failure modes of the GaBP algorithm can be understood via graph covers, and we prove that a parameterized generalization of the minsum algorithm can be used to ensure that the computation trees remain positive definite whenever the input matrix is positive definite. We demonstrate that the resulting algorithm is closely related to other iterative schemes for quadratic minimization such as the GaussSeidel and Jacobi algorithms. Finally, we observe, empirically, that there always exists a choice of parameters such that the above generalization of the GaBP algorithm converges.
 Publication:

arXiv eprints
 Pub Date:
 December 2012
 arXiv:
 arXiv:1212.0171
 Bibcode:
 2012arXiv1212.0171R
 Keywords:

 Computer Science  Information Theory;
 Computer Science  Machine Learning;
 Statistics  Machine Learning
 EPrint:
 Journal of Machine Learning Research. 14 (Aug) :22872314, 2013