Noise can speed convergence in Markov chains
Abstract
A new theorem shows that noise can speed convergence to equilibrium in discrete finitestate Markov chains. The noise applies to the state density and helps the Markov chain explore improbable regions of the state space. The theorem ensures that a stochasticresonance noise benefit exists for states that obey a vectornorm inequality. Such noise leads to faster convergence because the noise reduces the norm components. A corollary shows that a noise benefit still occurs if the system states obey an alternate norm inequality. This leads to a noisebenefit algorithm that requires knowledge of the steady state. An alternative blind algorithm uses only past state information to achieve a weaker noise benefit. Simulations illustrate the predicted noise benefits in three wellknown Markov models. The first model is a twoparameter Ehrenfest diffusion model that shows how noise benefits can occur in the class of birthdeath processes. The second model is a WrightFisher model of genotype drift in population genetics. The third model is a chemical reaction network of zeolite crystallization. A fourth simulation shows a convergence rate increase of 64% for states that satisfy the theorem and an increase of 53% for states that satisfy the corollary. A final simulation shows that even suboptimal noise can speed convergence if the noise applies over successive time cycles. Noise benefits tend to be sharpest in Markov models that do not converge quickly and that do not have strong absorbing states.
 Publication:

Physical Review E
 Pub Date:
 October 2011
 DOI:
 10.1103/PhysRevE.84.041112
 Bibcode:
 2011PhRvE..84d1112F
 Keywords:

 05.40.Ca;
 05.10.Ln;
 05.10.Gg;
 Noise;
 Monte Carlo methods;
 Stochastic analysis methods