A Priori Estimates of the Population Risk for Twolayer Neural Networks
Abstract
New estimates for the population risk are established for twolayer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the overparametrized regime when the network size is much larger than the size of the dataset. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model, in contrast with most existing results which are a posteriori in nature. Using these a priori estimates, we provide a perspective for understanding why twolayer neural networks perform better than the related kernel methods.
 Publication:

arXiv eprints
 Pub Date:
 October 2018
 arXiv:
 arXiv:1810.06397
 Bibcode:
 2018arXiv181006397E
 Keywords:

 Statistics  Machine Learning;
 Computer Science  Machine Learning;
 Mathematics  Statistics Theory;
 41A46;
 41A63;
 62J02;
 65D05
 EPrint:
 Published version