The Photometric LSST Astronomical Timeseries Classification Challenge PLAsTiCC: Selection of a Performance Metric for Classification Probabilities Balancing Diverse Science Goals
Abstract
Classification of transient and variable light curves is an essential step in using astronomical observations to develop an understanding of the underlying physical processes from which they arise. However, upcoming deep photometric surveys, including the Large Synoptic Survey Telescope (LSST), will produce a deluge of low signaltonoise data for which traditional type estimation procedures are inappropriate. Probabilistic classification is more appropriate for such data but is incompatible with the traditional metrics used on deterministic classifications. Furthermore, large survey collaborations like LSST intend to use the resulting classification probabilities for diverse science objectives, indicating a need for a metric that balances a variety of goals. We describe the process used to develop an optimal performance metric for an open classification challenge that seeks to identify probabilistic classifiers that can serve many scientific interests. The Photometric LSST Astronomical Timeseries Classification Challenge (PLASTICC) aims to identify promising techniques for obtaining classification probabilities of transient and variable objects by engaging a broader community beyond astronomy. Using mock classification probability submissions emulating realistically complex archetypes of those anticipated of PLASTICC, we compare the sensitivity of two metrics of classification probabilities under various weighting schemes, finding that both yield results that are qualitatively consistent with intuitive notions of classification performance. We thus choose as a metric for PLASTICC a weighted modification of the crossentropy because it can be meaningfully interpreted in terms of information content. Finally, we propose extensions of our methodology to ever more complex challenge goals and suggest some guiding principles for approaching the choice of a metric of probabilistic data products.
 Publication:

The Astronomical Journal
 Pub Date:
 November 2019
 DOI:
 10.3847/15383881/ab3a2f
 arXiv:
 arXiv:1809.11145
 Bibcode:
 2019AJ....158..171M
 Keywords:

 methods: data analysis;
 methods: statistical;
 stars: variables: general;
 supernovae: general;
 surveys;
 techniques: photometric;
 Astrophysics  Instrumentation and Methods for Astrophysics;
 Astrophysics  Cosmology and Nongalactic Astrophysics;
 Astrophysics  Solar and Stellar Astrophysics
 EPrint:
 doi:10.3847/15383881/ab3a2f