Tighter Variational Representations of f-Divergences via Restriction to Probability Measures
Abstract
We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on two i.i.d. samples and derive the dual program for this estimator that performs well empirically. We also point out a connection between our estimator and MMD.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2012
- DOI:
- 10.48550/arXiv.1206.4664
- arXiv:
- arXiv:1206.4664
- Bibcode:
- 2012arXiv1206.4664R
- Keywords:
-
- Computer Science - Machine Learning;
- Statistics - Machine Learning
- E-Print:
- ICML2012