Phase Transitions in Image Denoising via Sparsely Coding Convolutional Neural Networks
Abstract
Neural networks are analogous in many ways to spin glasses, systems which are known for their rich set of dynamics and equally complex phase diagrams. We apply well-known techniques in the study of spin glasses to a convolutional sparsely encoding neural network and observe power law finite-size scaling behavior in the sparsity and reconstruction error as the network denoises 32$\times$32 RGB CIFAR-10 images. This finite-size scaling indicates the presence of a continuous phase transition at a critical value of this sparsity. By using the power law scaling relations inherent to finite-size scaling, we can determine the optimal value of sparsity for any network size by tuning the system to the critical point and operate the system at the minimum denoising error.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2017
- DOI:
- 10.48550/arXiv.1710.09875
- arXiv:
- arXiv:1710.09875
- Bibcode:
- 2017arXiv171009875C
- Keywords:
-
- Computer Science - Neural and Evolutionary Computing;
- Condensed Matter - Statistical Mechanics;
- Computer Science - Computer Vision and Pattern Recognition
- E-Print:
- 4 pages, 3 figures, submitted to NIPS 2017 workshop: Advances in Modeling and Learning Interactions from Complex Data