Adapting Resilient Propagation for Deep Learning
Abstract
The Resilient Propagation (Rprop) algorithm has been very popular for backpropagation training of multilayer feed-forward neural networks in various applications. The standard Rprop however encounters difficulties in the context of deep neural networks as typically happens with gradient-based learning algorithms. In this paper, we propose a modification of the Rprop that combines standard Rprop steps with a special drop out technique. We apply the method for training Deep Neural Networks as standalone components and in ensemble formulations. Results on the MNIST dataset show that the proposed modification alleviates standard Rprop's problems demonstrating improved learning speed and accuracy.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2015
- DOI:
- 10.48550/arXiv.1509.04612
- arXiv:
- arXiv:1509.04612
- Bibcode:
- 2015arXiv150904612M
- Keywords:
-
- Computer Science - Neural and Evolutionary Computing;
- Computer Science - Computer Vision and Pattern Recognition;
- Computer Science - Machine Learning;
- Statistics - Machine Learning
- E-Print:
- Published in the proceedings of the UK workshop on Computational Intelligence 2015 (UKCI)