Renormalized Mutual Information for Artificial Scientific Discovery
Abstract
We derive a well-defined renormalized version of mutual information that allows us to estimate the dependence between continuous random variables in the important case when one is deterministically dependent on the other. This is the situation relevant for feature extraction, where the goal is to produce a low-dimensional effective description of a high-dimensional system. Our approach enables the discovery of collective variables in physical systems, thus adding to the toolbox of artificial scientific discovery, while also aiding the analysis of information flow in artificial neural networks.
- Publication:
-
Physical Review Letters
- Pub Date:
- May 2021
- DOI:
- 10.1103/PhysRevLett.126.200601
- arXiv:
- arXiv:2005.01912
- Bibcode:
- 2021PhRvL.126t0601S
- Keywords:
-
- Computer Science - Machine Learning;
- Physics - Data Analysis;
- Statistics and Probability
- E-Print:
- Added a more detailed introduction and link to code repository. Physics-based examples and Feature Extraction section have been updated