Online learning with ensembles
Abstract
Supervised online learning with an ensemble of students randomized by the choice of initial conditions is analyzed. For the case of the perceptron learning rule, asymptotically the same improvement in the generalization error of the ensemble compared to the performance of a single student is found as in Gibbs learning. For more optimized learning rules, however, using an ensemble yields no improvement. This is explained by showing that for any learning rule f a transform f~ exists, such that a single student using f~ has the same generalization behavior as an ensemble of f students.
- Publication:
-
Physical Review E
- Pub Date:
- July 2000
- DOI:
- 10.1103/PhysRevE.62.1448
- arXiv:
- arXiv:cond-mat/9907487
- Bibcode:
- 2000PhRvE..62.1448U
- Keywords:
-
- 87.10.+e;
- General theory and mathematical aspects;
- Condensed Matter - Disordered Systems and Neural Networks
- E-Print:
- 8 pages, 1 figure. Submitted to J.Phys.A