Non-convex Multi-species Hopfield Models
Abstract
In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern deep neural-network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton-Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the Mattis magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine with Gaussian hidden layer and the Bidirectional Associative Memory model.
- Publication:
-
Journal of Statistical Physics
- Pub Date:
- September 2018
- DOI:
- 10.1007/s10955-018-2098-6
- arXiv:
- arXiv:1807.03609
- Bibcode:
- 2018JSP...172.1247A
- Keywords:
-
- Neural networks;
- Boltzmann machines;
- Multipartite models;
- Condensed Matter - Disordered Systems and Neural Networks;
- Mathematical Physics
- E-Print:
- This is a pre-print of an article published in J. Stat. Phys