Scalable Bayesian model averaging through local information propagation
Abstract
We show that a probabilistic version of the classical forward-stepwise variable inclusion procedure can serve as a general data-augmentation scheme for model space distributions in (generalized) linear models. This latent variable representation takes the form of a Markov process, thereby allowing information propagation algorithms to be applied for sampling from model space posteriors. In particular, we propose a sequential Monte Carlo method for achieving effective unbiased Bayesian model averaging in high-dimensional problems, utilizing proposal distributions constructed using local information propagation. We illustrate our method---called LIPS for local information propagation based sampling---through real and simulated examples with dimensionality ranging from 15 to 1,000, and compare its performance in estimating posterior inclusion probabilities and in out-of-sample prediction to those of several other methods---namely, MCMC, BAS, iBMA, and LASSO. In addition, we show that the latent variable representation can also serve as a modeling tool for specifying model space priors that account for knowledge regarding model complexity and conditional inclusion relationships.
- Publication:
-
arXiv e-prints
- Pub Date:
- March 2014
- DOI:
- 10.48550/arXiv.1403.2397
- arXiv:
- arXiv:1403.2397
- Bibcode:
- 2014arXiv1403.2397M
- Keywords:
-
- Statistics - Methodology;
- Statistics - Computation;
- 62F15;
- 62J99;
- 62M05