Inference for Trans-dimensional Bayesian Models with Diffusive Nested Sampling
Abstract
Many inference problems involve inferring the number $N$ of components in some region, along with their properties $\{\mathbf{x}_i\}_{i=1}^N$, from a dataset $\mathcal{D}$. A common statistical example is finite mixture modelling. In the Bayesian framework, these problems are typically solved using one of the following two methods: i) by executing a Monte Carlo algorithm (such as Nested Sampling) once for each possible value of $N$, and calculating the marginal likelihood or evidence as a function of $N$; or ii) by doing a single run that allows the model dimension $N$ to change (such as Markov Chain Monte Carlo with birth/death moves), and obtaining the posterior for $N$ directly. In this paper we present a general approach to this problem that uses trans-dimensional MCMC embedded within a Nested Sampling algorithm, allowing us to explore the posterior distribution and calculate the marginal likelihood (summed over $N$) even if the problem contains a phase transition or other difficult features such as multimodality. We present two example problems, finding sinusoidal signals in noisy data, and finding and measuring galaxies in a noisy astronomical image. Both of the examples demonstrate phase transitions in the relationship between the likelihood and the cumulative prior mass, highlighting the need for Nested Sampling.
- Publication:
-
arXiv e-prints
- Pub Date:
- November 2014
- DOI:
- 10.48550/arXiv.1411.3921
- arXiv:
- arXiv:1411.3921
- Bibcode:
- 2014arXiv1411.3921B
- Keywords:
-
- Statistics - Computation;
- Astrophysics - Instrumentation and Methods for Astrophysics;
- Physics - Data Analysis;
- Statistics and Probability
- E-Print:
- Only published here for the time being. 17 pages, 10 figures. Software available at https://github.com/eggplantbren/RJObject