Cleaning large-dimensional covariance matrices for correlated samples
Abstract
We elucidate the problem of estimating large-dimensional covariance matrices in the presence of correlations between samples. To this end, we generalize the Marčenko-Pastur equation and the Ledoit-Péché shrinkage estimator using methods of random matrix theory and free probability. We develop an efficient algorithm that implements the corresponding analytic formulas based on the Ledoit-Wolf kernel estimation technique. We also provide an associated open-source Python library, called SHRINKAGE, with a user-friendly API to assist in practical tasks of estimation of large covariance matrices. We present an example of its usage for synthetic data generated according to exponentially decaying autocorrelations.
- Publication:
-
Physical Review E
- Pub Date:
- March 2022
- DOI:
- 10.1103/PhysRevE.105.034136
- arXiv:
- arXiv:2107.01352
- Bibcode:
- 2022PhRvE.105c4136B
- Keywords:
-
- Mathematical Physics;
- Mathematics - Statistics Theory;
- Quantitative Finance - Portfolio Management
- E-Print:
- 16 pages, 12 figures