Quantifying high-order interdependencies via multivariate extensions of the mutual information
Abstract
This paper introduces a model-agnostic approach to study statistical synergy, a form of emergence in which patterns at large scales are not traceable from lower scales. Our framework leverages various multivariate extensions of Shannon's mutual information, and introduces the O-information as a metric that is capable of characterizing synergy- and redundancy-dominated systems. The O-information is a symmetric quantity, and can assess intrinsic properties of a system without dividing its parts into "predictors" and "targets." We develop key analytical properties of the O-information, and study how it relates to other metrics of high-order interactions from the statistical mechanics and neuroscience literature. Finally, as a proof of concept, we present an exploration on the relevance of statistical synergy in Baroque music scores.
- Publication:
-
Physical Review E
- Pub Date:
- September 2019
- DOI:
- 10.1103/PhysRevE.100.032305
- arXiv:
- arXiv:1902.11239
- Bibcode:
- 2019PhRvE.100c2305R
- Keywords:
-
- Computer Science - Information Theory;
- Quantitative Biology - Neurons and Cognition
- E-Print:
- Phys. Rev. E 100, 032305 (2019)