Efficient approximation of high-dimensional functions with neural networks
Abstract
In this paper, we develop a framework for showing that neural networks can overcome the curse of dimensionality in different high-dimensional approximation problems. Our approach is based on the notion of a catalog network, which is a generalization of a standard neural network in which the nonlinear activation functions can vary from layer to layer as long as they are chosen from a predefined catalog of functions. As such, catalog networks constitute a rich family of continuous functions. We show that under appropriate conditions on the catalog, catalog networks can efficiently be approximated with rectified linear unit-type networks and provide precise estimates on the number of parameters needed for a given approximation accuracy. As special cases of the general results, we obtain different classes of functions that can be approximated with ReLU networks without the curse of dimensionality.
- Publication:
-
arXiv e-prints
- Pub Date:
- December 2019
- DOI:
- 10.48550/arXiv.1912.04310
- arXiv:
- arXiv:1912.04310
- Bibcode:
- 2019arXiv191204310C
- Keywords:
-
- Mathematics - Numerical Analysis;
- 68T07;
- I.2.0
- E-Print:
- IEEE Trans. Neural Netw. Learn. Syst. (2021)