Tight Lower Bounds for $\alpha$-Divergences Under Moment Constraints and Relations Between Different $\alpha$
Abstract
The $\alpha$-divergences include the well-known Kullback-Leibler divergence, Hellinger distance and $\chi^2$-divergence. In this paper, we derive differential and integral relations between the $\alpha$-divergences that are generalizations of the relation between the Kullback-Leibler divergence and the $\chi^2$-divergence. We also show tight lower bounds for the $\alpha$-divergences under given means and variances. In particular, we show a necessary and sufficient condition such that the binary divergences, which are divergences between probability measures on the same $2$-point set, always attain lower bounds. Kullback-Leibler divergence, Hellinger distance, and $\chi^2$-divergence satisfy this condition.
- Publication:
-
arXiv e-prints
- Pub Date:
- May 2021
- DOI:
- 10.48550/arXiv.2105.12972
- arXiv:
- arXiv:2105.12972
- Bibcode:
- 2021arXiv210512972N
- Keywords:
-
- Computer Science - Information Theory;
- Mathematics - Statistics Theory
- E-Print:
- 13 pages. arXiv admin note: text overlap with arXiv:2010.13548