Escaping Saddle Points in Ill-Conditioned Matrix Completion with a Scalable Second Order Method
Abstract
We propose an iterative algorithm for low-rank matrix completion that can be interpreted as both an iteratively reweighted least squares (IRLS) algorithm and a saddle-escaping smoothing Newton method applied to a non-convex rank surrogate objective. It combines the favorable data efficiency of previous IRLS approaches with an improved scalability by several orders of magnitude. Our method attains a local quadratic convergence rate already for a number of samples that is close to the information theoretical limit. We show in numerical experiments that unlike many state-of-the-art approaches, our approach is able to complete very ill-conditioned matrices with a condition number of up to $10^{10}$ from few samples.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2020
- DOI:
- 10.48550/arXiv.2009.02905
- arXiv:
- arXiv:2009.02905
- Bibcode:
- 2020arXiv200902905K
- Keywords:
-
- Mathematics - Optimization and Control;
- Computer Science - Information Theory;
- Computer Science - Machine Learning;
- Mathematics - Statistics Theory
- E-Print:
- 15 pages, presented at the Workshop on "Beyond first-order methods in ML systems" at the $37^th$ International Conference on Machine Learning (ICML), Vienna, Austria, 2020