Quantitative Weak Convergence for Discrete Stochastic Processes
Abstract
In this paper, we quantitative convergence in $W_2$ for a family of Langevin-like stochastic processes that includes stochastic gradient descent and related gradient-based algorithms. Under certain regularity assumptions, we show that the iterates of these stochastic processes converge to an invariant distribution at a rate of $\tilde{O}\lrp{1/\sqrt{k}}$ where $k$ is the number of steps; this rate is provably tight up to log factors. Our result reduces to a quantitative form of the classical Central Limit Theorem in the special case when the potential is quadratic.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2019
- DOI:
- 10.48550/arXiv.1902.00832
- arXiv:
- arXiv:1902.00832
- Bibcode:
- 2019arXiv190200832C
- Keywords:
-
- Mathematics - Statistics Theory;
- Computer Science - Machine Learning;
- Statistics - Machine Learning