Generalized Probability Smoothing
Abstract
In this work we consider a generalized version of Probability Smoothing, the core elementary model for sequential prediction in the state of the art PAQ family of data compression algorithms. Our main contribution is a code length analysis that considers the redundancy of Probability Smoothing with respect to a Piecewise Stationary Source. The analysis holds for a finite alphabet and expresses redundancy in terms of the total variation in probability mass of the stationary distributions of a Piecewise Stationary Source. By choosing parameters appropriately Probability Smoothing has redundancy $O(S\cdot\sqrt{T\log T})$ for sequences of length $T$ with respect to a Piecewise Stationary Source with $S$ segments.
- Publication:
-
arXiv e-prints
- Pub Date:
- December 2017
- DOI:
- 10.48550/arXiv.1712.02151
- arXiv:
- arXiv:1712.02151
- Bibcode:
- 2017arXiv171202151M
- Keywords:
-
- Computer Science - Information Theory