Variational (Gradient) Estimate of the Score Function in Energybased Latent Variable Models
Abstract
The learning and evaluation of energybased latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable. This paper presents variational estimates of the score function and its gradient with respect to the model parameters in a general EBLVM, referred to as VaES and VaGES respectively. The variational posterior is trained to minimize a certain divergence to the true model posterior and the bias in both estimates can be bounded by the divergence theoretically. With a minimal model assumption, VaES and VaGES can be applied to the kernelized Stein discrepancy (KSD) and score matching (SM)based methods to learn EBLVMs. Besides, VaES can also be used to estimate the exact Fisher divergence between the data and general EBLVMs.
 Publication:

arXiv eprints
 Pub Date:
 October 2020
 arXiv:
 arXiv:2010.08258
 Bibcode:
 2020arXiv201008258B
 Keywords:

 Computer Science  Machine Learning;
 Statistics  Machine Learning