Advanced mean-field theory of the restricted Boltzmann machine
Abstract
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean-field theory based on the Bethe approximation. Our theory provides an efficient message-passing-based method that evaluates not only the partition function (free energy) but also its gradients without requiring statistical sampling. The results are compared with those obtained by the computationally expensive sampling-based method.
- Publication:
-
Physical Review E
- Pub Date:
- May 2015
- DOI:
- 10.1103/PhysRevE.91.050101
- arXiv:
- arXiv:1502.00186
- Bibcode:
- 2015PhRvE..91e0101H
- Keywords:
-
- 02.50.Tt;
- 87.19.L-;
- 75.10.Nr;
- Inference methods;
- Neuroscience;
- Spin-glass and other random models;
- Condensed Matter - Statistical Mechanics;
- Computer Science - Machine Learning;
- Quantitative Biology - Neurons and Cognition;
- Statistics - Machine Learning
- E-Print:
- 5 pages, 4 figures, accepted by Phys Rev E (Rapid Communication)