Memory AMP
Abstract
Approximate message passing (AMP) is a lowcost iterative parameterestimation technique for certain highdimensional linear systems with nonGaussian distributions. AMP only applies to independent identically distributed (IID) transform matrices, but may become unreliable (e.g., perform poorly or even diverge) for other matrix ensembles, especially for illconditioned ones. To solve this issue, orthogonal/vector AMP (OAMP/VAMP) was proposed for general rightunitarilyinvariant matrices. However, the Bayesoptimal OAMP/VAMP (BOOAMP/VAMP) requires a highcomplexity linear minimum mean square error (MMSE) estimator. This prevents OAMP/VAMP from being used in largescale systems. To address the drawbacks of AMP and BOOAMP/VAMP, this paper offers a memory AMP (MAMP) framework based on the orthogonality principle, which ensures that estimation errors in MAMP are asymptotically IID Gaussian. To realize the required orthogonality for MAMP, we provide an orthogonalization procedure for the local memory estimators. In addition, we propose a Bayesoptimal MAMP (BOMAMP), in which a longmemory matched filter is used for interference suppression. The complexity of BOMAMP is comparable to AMP. To asymptotically characterize the performance of BOMAMP, a state evolution is derived. The relaxation parameters and damping vector in BOMAMP are optimized based on state evolution. Most crucially, the state evolution of the optimized BOMAMP converges to the same fixed point as that of the highcomplexity BOOAMP/VAMP for all rightunitarilyinvariant matrices, and achieves the Bayes optimal MSE predicted by the replica method if its state evolution has a unique fixed point. Finally, simulations are provided to verify the theoretical results' validity and accuracy.
 Publication:

arXiv eprints
 Pub Date:
 December 2020
 arXiv:
 arXiv:2012.10861
 Bibcode:
 2020arXiv201210861L
 Keywords:

 Computer Science  Information Theory;
 Computer Science  Artificial Intelligence;
 Computer Science  Machine Learning;
 Electrical Engineering and Systems Science  Signal Processing;
 Mathematics  Statistics Theory
 EPrint:
 Accepted by IEEE Trans. on Information Theory. [Memory AMP inherits the strengths of AMP and OAMP/VAMP such as low complexity, Bayes optimality and applicability to unitarilyinavariant matrices while avoiding the weakness of AMP (e.g. limited to IID matrices), OAMP/VAMP (e.g. needs highcomplexity LMMSE) and CAMP (e.g., Fails to converge in high condition numbers).]