Physical limits of inference
Abstract
We show that physical devices that perform observation, prediction, or recollection share an underlying mathematical structure. We call devices with that structure “inference devices”. We present a set of existence and impossibility results concerning inference devices. These results hold independent of the precise physical laws governing our universe. In a limited sense, the impossibility results establish that Laplace was wrong to claim that even in a classical, nonchaotic universe the future can be unerringly predicted, given sufficient knowledge of the present. Alternatively, these impossibility results can be viewed as a nonquantummechanical “uncertainty principle”. The mathematics of inference devices has close connections to the mathematics of Turing Machines (TMs). In particular, the impossibility results for inference devices are similar to the Halting theorem for TMs. Furthermore, one can define an analog of Universal TMs (UTMs) for inference devices. We call those analogs “strong inference devices”. We use strong inference devices to define the “inference complexity” of an inference task, which is the analog of the Kolmogorov complexity of computing a string. A taskindependent bound is derived on how much the inference complexity of an inference task can differ for two different inference devices. This is analogous to the “encoding” bound governing how much the Kolmogorov complexity of a string can differ between two UTMs used to compute that string. However no universe can contain more than one strong inference device. So whereas the Kolmogorov complexity of a string is arbitrary up to specification of the UTM, there is no such arbitrariness in the inference complexity of an inference task. We informally discuss the philosophical implications of these results, e.g., for whether the universe “is” a computer. We also derive some graphtheoretic properties governing any set of multiple inference devices. We also present an extension of the framework to address physical devices used for control. We end with an extension of the framework to address probabilistic inference.
 Publication:

Physica D Nonlinear Phenomena
 Pub Date:
 July 2008
 DOI:
 10.1016/j.physd.2008.03.040
 arXiv:
 arXiv:0708.1362
 Bibcode:
 2008PhyD..237.1257W
 Keywords:

 Condensed Matter  Statistical Mechanics;
 Computer Science  Computational Complexity;
 Computer Science  Information Theory;
 General Relativity and Quantum Cosmology
 EPrint:
 43 pages, updated version of Physica D version, which originally appeared in 2007 CNLS conference on unconventional computation