Multivariate fDivergence Estimation With Confidence
Abstract
The problem of fdivergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asymptotic distributions are unknown. We establish the asymptotic normality of a recently proposed ensemble estimator of fdivergence between two distributions from a finite number of samples. This estimator has MSE convergence rate of O(1/T), is simple to implement, and performs well in high dimensions. This theory enables us to perform divergencebased inference tasks such as testing equality of pairs of distributions based on empirical samples. We experimentally validate our theoretical results and, as an illustration, use them to empirically bound the best achievable classification error.
 Publication:

arXiv eprints
 Pub Date:
 November 2014
 arXiv:
 arXiv:1411.2045
 Bibcode:
 2014arXiv1411.2045M
 Keywords:

 Computer Science  Information Theory;
 Statistics  Machine Learning
 EPrint:
 20 pages, 1 figure. Accepted to NIPS 2014 (supplementary material is included in the appendices)