Optimal Variance Control of the Score Function Gradient Estimator for Importance Weighted Bounds
Abstract
This paper introduces novel results for the score function gradient estimator of the importance weighted variational bound (IWAE). We prove that in the limit of large $K$ (number of importance samples) one can choose the control variate such that the SignaltoNoise ratio (SNR) of the estimator grows as $\sqrt{K}$. This is in contrast to the standard pathwise gradient estimator where the SNR decreases as $1/\sqrt{K}$. Based on our theoretical findings we develop a novel control variate that extends on VIMCO. Empirically, for the training of both continuous and discrete generative models, the proposed method yields superior variance reduction, resulting in an SNR for IWAE that increases with $K$ without relying on the reparameterization trick. The novel estimator is competitive with stateoftheart reparameterizationfree gradient estimators such as Reweighted WakeSleep (RWS) and the thermodynamic variational objective (TVO) when training generative models.
 Publication:

arXiv eprints
 Pub Date:
 August 2020
 arXiv:
 arXiv:2008.01998
 Bibcode:
 2020arXiv200801998L
 Keywords:

 Statistics  Machine Learning;
 Computer Science  Machine Learning