Fits, and especially linear fits, with errors on both axes, extra variance of the data points and other complications

The aim of this paper, triggered by some discussions in the astrophysics community raised by astro-ph/0508529, is to introduce the issue of `fits' from a probabilistic perspective (also known as Bayesian), with special attention to the construction of model that describes the `network of dependences' (a Bayesian network) that connects experimental observations to model parameters and upon which the probabilistic inference relies. The particular case of linear fit with errors on both axes and extra variance of the data points around the straight line (i.e. not accounted by the experimental errors) is shown in detail. Some questions related to the use of linear fit formulas to log-linearized exponential and power laws are also sketched, as well as the issue of systematic errors.