Equivalence of approximation by convolutional neural networks and fully-connected networks
Abstract
Convolutional neural networks are the most widely used type of neural networks in applications. In mathematical analysis, however, mostly fully-connected networks are studied. In this paper, we establish a connection between both network architectures. Using this connection, we show that all upper and lower bounds concerning approximation rates of {fully-connected} neural networks for functions $f \in \mathcal{C}$ -- for an arbitrary function class $\mathcal{C}$ -- translate to essentially the same bounds concerning approximation rates of convolutional neural networks for functions $f \in {\mathcal{C}^{equi}}$, with the class ${\mathcal{C}^{equi}}$ consisting of all translation equivariant functions whose first coordinate belongs to $\mathcal{C}$. All presented results consider exclusively the case of convolutional neural networks without any pooling operation and with circular convolutions, i.e., not based on zero-padding.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2018
- DOI:
- 10.48550/arXiv.1809.00973
- arXiv:
- arXiv:1809.00973
- Bibcode:
- 2018arXiv180900973P
- Keywords:
-
- Mathematics - Functional Analysis;
- Computer Science - Machine Learning;
- 41A25;
- 44A35;
- 41A46
- E-Print:
- Proc. Amer. Math. Soc. 148 (2020), 1567-1581