The Asymptotic Distribution of the MLE in Highdimensional Logistic Models: Arbitrary Covariance
Abstract
We study the distribution of the maximum likelihood estimate (MLE) in highdimensional logistic models, extending the recent results from Sur (2019) to the case where the Gaussian covariates may have an arbitrary covariance structure. We prove that in the limit of large problems holding the ratio between the number $p$ of covariates and the sample size $n$ constant, every finite list of MLE coordinates follows a multivariate normal distribution. Concretely, the $j$th coordinate $\hat {\beta}_j$ of the MLE is asymptotically normally distributed with mean $\alpha_\star \beta_j$ and standard deviation $\sigma_\star/\tau_j$; here, $\beta_j$ is the value of the true regression coefficient, and $\tau_j$ the standard deviation of the $j$th predictor conditional on all the others. The numerical parameters $\alpha_\star > 1$ and $\sigma_\star$ only depend upon the problem dimensionality $p/n$ and the overall signal strength, and can be accurately estimated. Our results imply that the MLE's magnitude is biased upwards and that the MLE's standard deviation is greater than that predicted by classical theory. We present a series of experiments on simulated and real data showing excellent agreement with the theory.
 Publication:

arXiv eprints
 Pub Date:
 January 2020
 arXiv:
 arXiv:2001.09351
 Bibcode:
 2020arXiv200109351Z
 Keywords:

 Mathematics  Statistics Theory;
 Statistics  Methodology