Relative entropy via non-sequential recursive pair substitution

The entropy of an ergodic source is the limit of properly rescaled one-block entropies of sources obtained applying successive non-sequential recursive pair substitutions (NSRPS). In this paper we prove that the cross-entropy and the Kullback-Leibler divergence can be obtained in a similar way.