An Information-Theoretic Approach to Nonparametric Estimation, Model Selection, and Goodness of Fit
Abstract
This paper applies the recently axiomatized Optimum Information Principle (minimize the Kullback-Leibler information subject to all relevant information) to nonparametric density estimation, which provides a theoretical foundation as well as a computational algorithm for maximum entropy density estimation. The estimator, called optimum information estimator, approximates the true density arbitrarily well. As a by-product I obtain a measure of goodness of fit of parametric models (both conditional and unconditional) and an absolute criterion for model selection, as opposed to other conventional methods such as AIC and BIC which are relative measures.
- Publication:
-
arXiv e-prints
- Pub Date:
- March 2011
- DOI:
- 10.48550/arXiv.1103.4890
- arXiv:
- arXiv:1103.4890
- Bibcode:
- 2011arXiv1103.4890A
- Keywords:
-
- Mathematics - Statistics Theory;
- Statistics - Methodology
- E-Print:
- Submitted to Econometrica on March 24, 2011