Multi-class SVMs: From Tighter Data-Dependent Generalization Bounds to Novel Algorithms
Abstract
This paper studies the generalization performance of multi-class classification algorithms, for which we obtain, for the first time, a data-dependent generalization error bound with a logarithmic dependence on the class size, substantially improving the state-of-the-art linear dependence in the existing data-dependent generalization analysis. The theoretical analysis motivates us to introduce a new multi-class classification machine based on $\ell_p$-norm regularization, where the parameter $p$ controls the complexity of the corresponding bounds. We derive an efficient optimization algorithm based on Fenchel duality theory. Benchmarks on several real-world datasets show that the proposed algorithm can achieve significant accuracy gains over the state of the art.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2015
- DOI:
- 10.48550/arXiv.1506.04359
- arXiv:
- arXiv:1506.04359
- Bibcode:
- 2015arXiv150604359L
- Keywords:
-
- Computer Science - Machine Learning