On Constructing Confidence Region for Model Parameters in Stochastic Gradient Descent via Batch Means
Abstract
In this paper, we study a simple algorithm to construct asymptotically valid confidence regions for model parameters using the batch means method. The main idea is to cancel out the covariance matrix which is hard/costly to estimate. In the process of developing the algorithm, we establish process-level functional central limit theorem for Polyak-Ruppert averaging based stochastic gradient descent estimators. We also extend the batch means method to accommodate more general batch size specifications.
- Publication:
-
arXiv e-prints
- Pub Date:
- November 2019
- DOI:
- 10.48550/arXiv.1911.01483
- arXiv:
- arXiv:1911.01483
- Bibcode:
- 2019arXiv191101483Z
- Keywords:
-
- Statistics - Machine Learning;
- Computer Science - Machine Learning;
- Mathematics - Statistics Theory