OptShrink: An algorithm for improved lowrank signal matrix denoising by optimal, datadriven singular value shrinkage
Abstract
The truncated singular value decomposition (SVD) of the measurement matrix is the optimal solution to the_representation_ problem of how to best approximate a noisy measurement matrix using a lowrank matrix. Here, we consider the (unobservable)_denoising_ problem of how to best approximate a lowrank signal matrix buried in noise by optimal (re)weighting of the singular vectors of the measurement matrix. We exploit recent results from random matrix theory to exactly characterize the large matrix limit of the optimal weighting coefficients and show that they can be computed directly from data for a large class of noise models that includes the i.i.d. Gaussian noise case. Our analysis brings into sharp focus the shrinkageandthresholding form of the optimal weights, the nonconvex nature of the associated shrinkage function (on the singular values) and explains why matrix regularization via singular value thresholding with convex penalty functions (such as the nuclear norm) will always be suboptimal. We validate our theoretical predictions with numerical simulations, develop an implementable algorithm (OptShrink) that realizes the predicted performance gains and show how our methods can be used to improve estimation in the setting where the measured matrix has missing entries.
 Publication:

arXiv eprints
 Pub Date:
 June 2013
 arXiv:
 arXiv:1306.6042
 Bibcode:
 2013arXiv1306.6042R
 Keywords:

 Mathematics  Statistics Theory;
 Computer Science  Information Theory;
 Statistics  Machine Learning
 EPrint:
 Published version. The algorithm can be downloaded from http://www.eecs.umich.edu/~rajnrao/optshrink