Estimating the number of species to attain sufficient representation in a random sample
Abstract
The statistical problem of using an initial sample to estimate the number of species in a larger sample has found important applications in fields far removed from ecology. Here we address the general problem of estimating the number of species that will be represented by at least a number r of observations in a future sample. The number r indicates species with sufficient observations, which are commonly used as a necessary condition for any robust statistical inference. We derive a procedure to construct consistent estimators that apply universally for a given population: once constructed, they can be evaluated as a simple function of r. Our approach is based on a relation between the number of species represented at least r times and the higher derivatives of the expected number of species discovered per unit of time. Combining this relation with a rational function approximation, we propose nonparametric estimators that are accurate for both large values of r and longrange extrapolations. We further show that our estimators retain asymptotic behaviors that are essential for applications on largescale datasets. We evaluate the performance of this approach by both simulation and real data applications for inferences of the vocabulary of Shakespeare and Dickens, the topology of a Twitter social network, and molecular diversity in DNA sequencing data.
 Publication:

arXiv eprints
 Pub Date:
 July 2016
 arXiv:
 arXiv:1607.02804
 Bibcode:
 2016arXiv160702804D
 Keywords:

 Statistics  Methodology
 EPrint:
 19 pages, 5 figures, 3 tables