megaman: Manifold Learning with Millions of points
Abstract
Manifold Learning is a class of algorithms seeking a lowdimensional nonlinear representation of highdimensional data. Thus manifold learning algorithms are, at least in theory, most applicable to highdimensional data and sample sizes to enable accurate estimation of the manifold. Despite this, most existing manifold learning implementations are not particularly scalable. Here we present a Python package that implements a variety of manifold learning algorithms in a modular and scalable fashion, using fast approximate neighbors searches and fast sparse eigendecompositions. The package incorporates theoretical advances in manifold learning, such as the unbiased Laplacian estimator and the estimation of the embedding distortion by the Riemannian metric method. In benchmarks, even on a singlecore desktop computer, our code embeds millions of data points in minutes, and takes just 200 minutes to embed the main sample of galaxy spectra from the Sloan Digital Sky Survey  consisting of 0.6 million samples in 3750dimensions  a task which has not previously been possible.
 Publication:

arXiv eprints
 Pub Date:
 March 2016
 arXiv:
 arXiv:1603.02763
 Bibcode:
 2016arXiv160302763M
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Computational Geometry;
 Statistics  Machine Learning
 EPrint:
 12 pages, 6 figures