Randomized Riemannian Preconditioning for Orthogonality Constrained Problems
Abstract
Optimization problems with (generalized) orthogonality constraints are prevalent across science and engineering. For example, in computational science they arise in the symmetric (generalized) eigenvalue problem, in nonlinear eigenvalue problems, and in electronic structures computations, to name a few problems. In statistics and machine learning, they arise, for example, in canonical correlation analysis and in linear discriminant analysis. In this article, we consider using randomized preconditioning in the context of optimization problems with generalized orthogonality constraints. Our proposed algorithms are based on Riemannian optimization on the generalized Stiefel manifold equipped with a nonstandard preconditioned geometry, which necessitates development of the geometric components necessary for developing algorithms based on this approach. Furthermore, we perform asymptotic convergence analysis of the preconditioned algorithms which help to characterize the quality of a given preconditioner using secondorder information. Finally, for the problems of canonical correlation analysis and linear discriminant analysis, we develop randomized preconditioners along with corresponding bounds on the relevant condition number.
 Publication:

arXiv eprints
 Pub Date:
 February 2019
 arXiv:
 arXiv:1902.01635
 Bibcode:
 2019arXiv190201635S
 Keywords:

 Mathematics  Numerical Analysis;
 Computer Science  Data Structures and Algorithms;
 Computer Science  Machine Learning