MMSE of probabilistic lowrank matrix estimation: Universality with respect to the output channel
Abstract
This paper considers probabilistic estimation of a lowrank matrix from nonlinear elementwise measurements of its elements. We derive the corresponding approximate message passing (AMP) algorithm and its state evolution. Relying on nonrigorous but standard assumptions motivated by statistical physics, we characterize the minimum mean squared error (MMSE) achievable information theoretically and with the AMP algorithm. Unlike in related problems of linear estimation, in the present setting the MMSE depends on the output channel only trough a single parameter  its Fisher information. We illustrate this striking finding by analysis of submatrix localization, and of detection of communities hidden in a dense stochastic block model. For this example we locate the computational and statistical boundaries that are not equal for rank larger than four.
 Publication:

arXiv eprints
 Pub Date:
 July 2015
 arXiv:
 arXiv:1507.03857
 Bibcode:
 2015arXiv150703857L
 Keywords:

 Computer Science  Information Theory;
 Condensed Matter  Statistical Mechanics;
 Statistics  Machine Learning
 EPrint:
 10 pages, Allerton Conference on Communication, Control, and Computing 2015