Exploring epoch-dependent stochastic residual networks
Abstract
The recently proposed stochastic residual networks selectively activate or bypass the layers during training, based on independent stochastic choices, each of which following a probability distribution that is fixed in advance. In this paper we present a first exploration on the use of an epoch-dependent distribution, starting with a higher probability of bypassing deeper layers and then activating them more frequently as training progresses. Preliminary results are mixed, yet they show some potential of adding an epoch-dependent management of distributions, worth of further investigation.
- Publication:
-
arXiv e-prints
- Pub Date:
- April 2017
- DOI:
- 10.48550/arXiv.1704.06178
- arXiv:
- arXiv:1704.06178
- Bibcode:
- 2017arXiv170406178C
- Keywords:
-
- Computer Science - Computer Vision and Pattern Recognition
- E-Print:
- Preliminary report