Nonextensive Generalizations of the Jensen-Shannon Divergence
Abstract
Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon divergence (JSD), which is a symmetrized and smoothed version of the KLD. This paper introduces new JSD-type divergences, by extending its two building blocks: convexity and Shannon's entropy. In particular, a new concept of q-convexity is introduced and shown to satisfy a Jensen's q-inequality. Based on this Jensen's q-inequality, the Jensen-Tsallis q-difference is built, which is a nonextensive generalization of the JSD, based on Tsallis entropies. Finally, the Jensen-Tsallis q-difference is charaterized in terms of convexity and extrema.
- Publication:
-
arXiv e-prints
- Pub Date:
- April 2008
- DOI:
- 10.48550/arXiv.0804.1653
- arXiv:
- arXiv:0804.1653
- Bibcode:
- 2008arXiv0804.1653M
- Keywords:
-
- Computer Science - Information Theory;
- Mathematics - Statistics;
- 94A17;
- 94A15
- E-Print:
- Submitted to the IEEE Transactions on Information Theory