Statistical Guarantees for Regularized Neural Networks
Abstract
Neural networks have become standard tools in the analysis of data, but they lack comprehensive mathematical theories. For example, there are very few statistical guarantees for learning neural networks from data, especially for classes of estimators that are used in practice or at least similar to such. In this paper, we develop a general statistical guarantee for estimators that consist of a leastsquares term and a regularizer. We then exemplify this guarantee with $\ell_1$regularization, showing that the corresponding prediction error increases at most sublinearly in the number of layers and at most logarithmically in the total number of parameters. Our results establish a mathematical basis for regularized estimation of neural networks, and they deepen our mathematical understanding of neural networks and deep learning more generally.
 Publication:

arXiv eprints
 Pub Date:
 May 2020
 arXiv:
 arXiv:2006.00294
 Bibcode:
 2020arXiv200600294T
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Neural and Evolutionary Computing;
 Mathematics  Statistics Theory;
 Statistics  Methodology;
 Statistics  Machine Learning