How Many Iterations are Sufficient for Semiparametric Estimation?
Abstract
A common practice in obtaining a semiparametric efficient estimate is through iteratively maximizing the (penalized) log-likelihood w.r.t. its Euclidean parameter and functional nuisance parameter via Newton-Raphson algorithm. The purpose of this paper is to provide a formula in calculating the minimal number of iterations $k^\ast$ needed to produce an efficient estimate $\hat\theta_n^{(k^\ast)}$ from a theoretical point of view. We discover that (a) $k^\ast$ depends on the convergence rates of the initial estimate and nuisance estimate; (b) more than $k^\ast$ iterations, i.e., $k$, will only improve the higher order asymptotic efficiency of $\hat\theta_n^{(k)}$; (c) $k^\ast$ iterations are also sufficient for recovering the estimation sparsity in high dimensional data. These general conclusions hold, in particular, when the nuisance parameter is not estimable at root-n rate, and apply to semiparametric models estimated under various regularizations, e.g., kernel or penalized estimation. This paper provides a first general theoretical justification for the "one-/two-step iteration" phenomena observed in the literature, and may be useful in reducing the bootstrap computational cost for the semiparametric models.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2010
- DOI:
- 10.48550/arXiv.1009.2111
- arXiv:
- arXiv:1009.2111
- Bibcode:
- 2010arXiv1009.2111C
- Keywords:
-
- Mathematics - Statistics Theory
- E-Print:
- 42 pages, submitted to the Annals of Statistics