Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks
Abstract
We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of $n$ output units, given the states of $k\geq1$ input units, can be approximated arbitrarily well by a network with $2^{k-1}(2^{n-1}-1)$ hidden units.
- Publication:
-
arXiv e-prints
- Pub Date:
- March 2015
- DOI:
- 10.48550/arXiv.1503.07211
- arXiv:
- arXiv:1503.07211
- Bibcode:
- 2015arXiv150307211M
- Keywords:
-
- Computer Science - Machine Learning;
- Statistics - Machine Learning;
- 82C32
- E-Print:
- 13 pages, 3 figures