Generalization properties of finite-size polynomial support vector machines
Abstract
The learning properties of finite-size polynomial support vector machines are analyzed in the case of realizable classification tasks. The normalization of the high-order features acts as a squeezing factor, introducing a strong anisotropy in the patterns distribution in feature space. As a function of the training set size, the corresponding generalization error presents a crossover, more or less abrupt depending on the distribution's anisotropy and on the task to be learned, between a fast-decreasing and a slowly decreasing regime. This behavior corresponds to the stepwise decrease found by Dietrich et al. [Phys. Rev. Lett. 82, 2975 (1999)] in the thermodynamic limit. The theoretical results are in excellent agreement with the numerical simulations.
- Publication:
-
Physical Review E
- Pub Date:
- November 2000
- DOI:
- 10.1103/PhysRevE.62.7092
- arXiv:
- arXiv:cond-mat/0002071
- Bibcode:
- 2000PhRvE..62.7092R
- Keywords:
-
- 87.10.+e;
- 02.50.-r;
- 05.20.-y;
- General theory and mathematical aspects;
- Probability theory stochastic processes and statistics;
- Classical statistical mechanics;
- Condensed Matter - Disordered Systems and Neural Networks;
- Condensed Matter - Statistical Mechanics
- E-Print:
- 12 pages, 7 figures