Large Alphabets and Incompressibility
Abstract
We briefly survey some concepts related to empirical entropy -- normal numbers, de Bruijn sequences and Markov processes -- and investigate how well it approximates Kolmogorov complexity. Our results suggest $\ell$th-order empirical entropy stops being a reasonable complexity metric for almost all strings of length $m$ over alphabets of size $n$ about when $n^\ell$ surpasses $m$.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2005
- DOI:
- 10.48550/arXiv.cs/0506056
- arXiv:
- arXiv:cs/0506056
- Bibcode:
- 2005cs........6056G
- Keywords:
-
- Computer Science - Information Theory;
- E.4
- E-Print:
- doi:10.1016/j.ipl.2006.04.008