Deep Boltzmann Machines: Rigorous Results at Arbitrary Depth
Abstract
A class of deep Boltzmann machines is considered in the simplified framework of a quenched system with Gaussian noise and independent entries. The quenched pressure of a $K$-layers spin glass model is studied allowing interactions only among consecutive layers. A lower bound for the pressure is found in terms of a convex combination of $K$ Sherrington-Kirkpatrick models and used to study the annealed and replica symmetric regimes of the system. A map with a one dimensional monomer-dimer system is identified and used to rigorously control the annealed region at arbitrary depth $K$ with the methods introduced by Heilmann and Lieb. The compression of this high noise region displays a remarkable phenomenon of localisation of the processing layers. Furthermore a replica symmetric lower bound for the limiting quenched pressure of the model is obtained in a suitable region of the parameters and the replica symmetric pressure is proved to have a unique stationary point.
- Publication:
-
Annales Henri Poincaré
- Pub Date:
- August 2021
- DOI:
- 10.1007/s00023-021-01027-2
- arXiv:
- arXiv:2004.04495
- Bibcode:
- 2021AnHP...22.2619A
- Keywords:
-
- Mathematical Physics;
- Condensed Matter - Disordered Systems and Neural Networks
- E-Print:
- 28 pages