Stochastic Backpropagation and Approximate Inference in Deep Generative Models
Abstract
We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation -- rules for back-propagation through stochastic variables -- and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.
- Publication:
-
arXiv e-prints
- Pub Date:
- January 2014
- DOI:
- 10.48550/arXiv.1401.4082
- arXiv:
- arXiv:1401.4082
- Bibcode:
- 2014arXiv1401.4082J
- Keywords:
-
- Statistics - Machine Learning;
- Computer Science - Artificial Intelligence;
- Computer Science - Machine Learning;
- Statistics - Computation;
- Statistics - Methodology
- E-Print:
- Appears In Proceedings of the 31st International Conference on Machine Learning (ICML), JMLR: W\&