A Bayesian framework for causality estimation

Estimates of causal relations will be uncertain, and a natural way to formulate this uncertainty is through a probability density function (pdf). We explore Bayes Theorem to describes how our prior pdf of a causal strength cs between two processes changes when new observational information becomes available, as:

p(cs | cm) = p(cs) p(cm | cs) / p(cm)

in which p(..) is the probability density function, cs is the causality strength, and cm is the value of the causality measure from the observations. The goal is to find an estimate of the actual causality strength including its uncertainty in the form of a pdf, given all information available. One of the great advantages of Bayes Theorem is that it is a learning framework, so that If new information becomes available we can use this equation again, in which the previous posterior pdf of causality strength becomes our new prior pdf.

To use this framework we need to

1) Define the causality strength. It will be proportional to the fraction of entropy reduction of process x by knowledge of the history of process y, calculated via normalised conditional mutual information.

2) Provide the prior pdf of causality strength p(cs). If no prior information is available we start with a flat pdf, but prior knowledge from experts can also be used.

3) Calculate the likelihood p(cm | cs), so determine the accuracy of our measure of causality cm. The estimate is inaccurate due to finite sample size, and the estimate can be incomplete, because we are missing a process in the calculation of the causality chain. Several ways to determine this have been proposed.

I will explain the framework in detail and provide examples of its application.