Network Inference and Maximum Entropy Estimation on Information Diagrams
Abstract
Maximum entropy estimation is of broad interest for inferring properties of systems across many disciplines. Using a recently introduced technique for estimating the maximum entropy of a set of random discrete variables when conditioning on bivariate mutual informations and univariate entropies, we show how this can be used to estimate the direct network connectivity between interacting units from observed activity. As a generic example, we consider phase oscillators and show that our approach is typically superior to simply using the mutual information. In addition, we propose a nonparametric formulation of connected informations, used to test the explanatory power of a network description in general. We give an illustrative example showing how this agrees with the existing parametric formulation, and demonstrate its applicability and advantages for restingstate human brain networks, for which we also discuss its direct effective connectivity. Finally, we generalize to continuous random variables and vastly expand the types of informationtheoretic quantities one can condition on. This allows us to establish significant advantages of this approach over existing ones. Not only does our method perform favorably in the undersampled regime, where existing methods fail, but it also can be dramatically less computationally expensive as the cardinality of the variables increases.
 Publication:

Scientific Reports
 Pub Date:
 August 2017
 DOI:
 10.1038/s4159801706208w
 arXiv:
 arXiv:1601.00336
 Bibcode:
 2017NatSR...7.7062M
 Keywords:

 Physics  Data Analysis;
 Statistics and Probability