Adapting to Non-stationarity with Growing Expert Ensembles
Abstract
When dealing with time series with complex non-stationarities, low retrospective regret on individual realizations is a more appropriate goal than low prospective risk in expectation. Online learning algorithms provide powerful guarantees of this form, and have often been proposed for use with non-stationary processes because of their ability to switch between different forecasters or ``experts''. However, existing methods assume that the set of experts whose forecasts are to be combined are all given at the start, which is not plausible when dealing with a genuinely historical or evolutionary system. We show how to modify the ``fixed shares'' algorithm for tracking the best expert to cope with a steadily growing set of experts, obtained by fitting new models to new data as it becomes available, and obtain regret bounds for the growing ensemble.
- Publication:
-
arXiv e-prints
- Pub Date:
- March 2011
- DOI:
- 10.48550/arXiv.1103.0949
- arXiv:
- arXiv:1103.0949
- Bibcode:
- 2011arXiv1103.0949R
- Keywords:
-
- Statistics - Machine Learning;
- Computer Science - Machine Learning;
- Physics - Data Analysis;
- Statistics and Probability;
- Statistics - Methodology
- E-Print:
- 9 pages, 1 figure