Source Coding, Large Deviations, and Approximate Pattern Matching
Abstract
We present a development of parts of ratedistortion theory and pattern matching algorithms for lossy data compression, centered around a lossy version of the Asymptotic Equipartition Property (AEP). This treatment closely parallels the corresponding development in lossless compression, a point of view that was advanced in an important paper of Wyner and Ziv in 1989. In the lossless case we review how the AEP underlies the analysis of the LempelZiv algorithm by viewing it as a random code and reducing it to the idealized Shannon code. This also provides information about the redundancy of the LempelZiv algorithm and about the asymptotic behavior of several relevant quantities. In the lossy case we give various versions of the statement of the generalized AEP and we outline the general methodology of its proof via large deviations. Its relationship with Barron's generalized AEP is also discussed. The lossy AEP is applied to: (i) prove strengthened versions of Shannon's source coding theorem and universal coding theorems; (ii) characterize the performance of mismatched codebooks; (iii) analyze the performance of pattern matching algorithms for lossy compression; (iv) determine the first order asymptotics of waiting times (with distortion) between stationary processes; (v) characterize the best achievable rate of weighted codebooks as an optimal spherecovering exponent. We then present a refinement to the lossy AEP and use it to: (i) prove second order coding theorems; (ii) characterize which sources are easier to compress; (iii) determine the second order asymptotics of waiting times; (iv) determine the precise asymptotic behavior of longest matchlengths. Extensions to random fields are also given.
 Publication:

arXiv Mathematics eprints
 Pub Date:
 March 2001
 arXiv:
 arXiv:math/0103007
 Bibcode:
 2001math......3007D
 Keywords:

 Mathematics  Probability;
 Computer Science  Information Theory;
 94A15;
 60F10;
 60G60
 EPrint:
 48 pages, review paper