Minimal penalties and the slope heuristics: a survey
Abstract
Birg{é} and Massart proposed in 2001 the slope heuristics as a way to choose optimally from data an unknown multiplicative constant in front of a penalty. It is built upon the notion of minimal penalty, and it has been generalized since to some "minimal-penalty algorithms". This paper reviews the theoretical results obtained for such algorithms, with a self-contained proof in the simplest framework, precise proof ideas for further generalizations, and a few new results. Explicit connections are made with residual-variance estimators-with an original contribution on this topic, showing that for this task the slope heuristics performs almost as well as a residual-based estimator with the best model choice-and some classical algorithms such as L-curve or elbow heuristics, Mallows' C p , and Akaike's FPE. Practical issues are also addressed, including two new practical definitions of minimal-penalty algorithms that are compared on synthetic data to previously-proposed definitions. Finally, several conjectures and open problems are suggested as future research directions.
- Publication:
-
arXiv e-prints
- Pub Date:
- January 2019
- DOI:
- 10.48550/arXiv.1901.07277
- arXiv:
- arXiv:1901.07277
- Bibcode:
- 2019arXiv190107277A
- Keywords:
-
- Mathematics - Statistics Theory;
- Statistics - Methodology;
- Statistics - Machine Learning
- E-Print:
- Journal de la Societe Fran{\c c}aise de Statistique, Societe Fran{\c c}aise de Statistique et Societe Mathematique de France, 2019, Minimal penalties and the slope heuristics: a survey, 160 (3), pp.1-106