Uniform Deviation Bounds for Unbounded Loss Functions like k-Means
Abstract
Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss functions which are *unbounded*. In our main application, this allows us to obtain bounds for $k$-Means clustering under weak assumptions on the underlying distribution. If the fourth moment is bounded, we prove a rate of $\mathcal{O}\left(m^{-\frac12}\right)$ compared to the previously known $\mathcal{O}\left(m^{-\frac14}\right)$ rate. Furthermore, we show that the rate also depends on the kurtosis - the normalized fourth moment which measures the "tailedness" of a distribution. We further provide improved rates under progressively stronger assumptions, namely, bounded higher moments, subgaussianity and bounded support.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2017
- DOI:
- 10.48550/arXiv.1702.08249
- arXiv:
- arXiv:1702.08249
- Bibcode:
- 2017arXiv170208249B
- Keywords:
-
- Statistics - Machine Learning;
- Computer Science - Machine Learning