Multi-variate correlation and mixtures of product measures
Abstract
Total correlation (`TC') and dual total correlation (`DTC') are two classical ways to quantify the correlation among an $n$-tuple of random variables. They both reduce to mutual information when $n=2$. The first part of this paper sets up the theory of TC and DTC for general random variables, not necessarily finite-valued. This generality has not been exposed in the literature before. The second part considers the structural implications when a joint distribution $\mu$ has small TC or DTC. If $\mathrm{TC}(\mu) = o(n)$, then $\mu$ is close to a product measure according to a suitable transportation metric: this follows directly from Marton's classical transportation-entropy inequality. If $\mathrm{DTC}(\mu) = o(n)$, then the structural consequence is more complicated: $\mu$ is a mixture of a controlled number of terms, most of them close to product measures in the transportation metric. This is the main new result of the paper.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2018
- DOI:
- 10.48550/arXiv.1809.10272
- arXiv:
- arXiv:1809.10272
- Bibcode:
- 2018arXiv180910272A
- Keywords:
-
- Mathematics - Probability;
- Computer Science - Information Theory;
- Mathematics - Statistics Theory;
- 60B99 (primary);
- 60G99;
- 62B10;
- 94A17
- E-Print:
- 39 pages [v2:] Slight changes in presentation based on feedback from colleagues [v3, v4:] Small revisions during preparation for journal