Hierachical Bayesian models and sparsity: L2-magic

Sparse recovery seeks to estimate the support and the non-zero entries of a sparse signal x ∈ R ⁿ from incomplete noisy observations y = mA x₀₊ ɛ , with mA \ R ^{m × n}, m <n. It has been shown that under various restrictive conditions on the matrix mA, the problem can be reduced to the ℓ₁ regularized problem [ \min \|x\|_{1} ; subject to ; \|\mA x - y \|_2 < δ, ] where δ is the size of the error ɛ , and the approximation error is well controlled by δ . A popular algorithm of solving the above minimization problem is the iteratively reweighted least squares algorithm. Here we reformulate the question of sparse recovery as an inverse problems in the Bayesian framework, express the sparsity belief by means of a hierachical prior model and show that the Maximum a Posteriori (MAP) solution computed by a recently proposed Iterative Alternating Sequential (IAS) algorithm requiring only the solution of linear systems in the least squares sense converges linearly to the unique minimum for any matrix mA, and quadratically on the complement of the support of the minimizer. The values of the parameters of the hierarchical model are assigned from an estimate of the signal to noise ratio and a priori belief of the degree of sparsity of the underlying signal, and automatically take into account the sensitivity of the data to the different components of x. The approach gives a solid Bayesian interpretation for the commonly used sensitivity weighting in geophysics and biomedical applications. Moreover, since for a suitable choice of sequences of parameters of the hyperprior, the IAS solution converges to the ℓ₁ regularized solution, the Bayesian framework for inverse problems makes the ℓ₁-magic happen in the ℓ₂ framework.