A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)
Abstract
The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description Length and error minimization. We conclude with a discussion of the typical features of overfitting in model selection.
- Publication:
-
arXiv e-prints
- Pub Date:
- May 2010
- DOI:
- 10.48550/arXiv.1005.2364
- arXiv:
- arXiv:1005.2364
- Bibcode:
- 2010arXiv1005.2364N
- Keywords:
-
- Computer Science - Machine Learning;
- Computer Science - Computational Complexity;
- F.2.3
- E-Print:
- 20 pages, Chapter 1 of The Paradox of Overfitting, Master's thesis, Rijksuniversiteit Groningen, 2003