Finite Littlestone Dimension Implies Finite Information Complexity
Abstract
We prove that every online learnable class of functions of Littlestone dimension $d$ admits a learning algorithm with finite information complexity. Towards this end, we use the notion of a globally stable algorithm. Generally, the information complexity of such a globally stable algorithm is large yet finite, roughly exponential in $d$. We also show there is room for improvement; for a canonical online learnable class, indicator functions of affine subspaces of dimension $d$, the information complexity can be upper bounded logarithmically in $d$.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2022
- DOI:
- 10.48550/arXiv.2206.13257
- arXiv:
- arXiv:2206.13257
- Bibcode:
- 2022arXiv220613257P
- Keywords:
-
- Computer Science - Machine Learning;
- Computer Science - Information Theory