Principal Component Analysis with Noisy and/or Missing Data
Abstract
We present a method for performing principal component analysis (PCA) on noisy datasets with missing values. Estimates of the measurement error are used to weight the input data such that the resulting eigenvectors, when compared to classic PCA, are more sensitive to the true underlying signal variations rather than being pulled by heteroskedastic measurement noise. Missing data are simply limiting cases of weight = 0. The underlying algorithm is a noise weighted expectation maximization (EM) PCA, which has additional benefits of implementation speed and flexibility for smoothing eigenvectors to reduce the noise contribution. We present applications of this method on simulated data and QSO spectra from the Sloan Digital Sky Survey (SDSS).
- Publication:
-
Publications of the Astronomical Society of the Pacific
- Pub Date:
- September 2012
- DOI:
- 10.1086/668105
- arXiv:
- arXiv:1208.4122
- Bibcode:
- 2012PASP..124.1015B
- Keywords:
-
- Astrophysics - Instrumentation and Methods for Astrophysics;
- Physics - Data Analysis;
- Statistics and Probability
- E-Print:
- Accepted for publication in PASP