Cooperative Training of Descriptor and Generator Networks
Abstract
This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energybased model, whose energy function is defined by a bottomup ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a nonlinear version of factor analysis. It is defined by a topdown ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finitestep MCMC that samples and trains the energybased descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.
 Publication:

arXiv eprints
 Pub Date:
 September 2016
 arXiv:
 arXiv:1609.09408
 Bibcode:
 2016arXiv160909408X
 Keywords:

 Statistics  Machine Learning;
 Computer Science  Computer Vision and Pattern Recognition
 EPrint:
 18 pages