The Photometric LSST Astronomical Time Series Classification Challenge (PLAsTiCC): challenge design and evaluation criteria
Abstract
PLAsTiCC differs from previous astronomical classification challenges in three ways that complicate the conventionally simple question of, "Who should win?" First, because of the limited information anticipated of individual LSST-like light curves, classification submissions will be probability vectors, not labels; in the absence of certainty about whether a label is "right" or "wrong" we considered more nuanced performance metrics. Second, the winning classifier must serve the needs of a diverse community of scientists that will use the classification probabilities to answer myriad questions about the universe; a classifier that focuses on one class and ignores all others would be inappropriate, so we devised a way to level the playing field by penalizing neglect of any classes. Third, PLAsTiCC's engagement with competitors outside astronomy is reciprocated by the engagement of non-astronomers with astronomical data; we investigated how to disincentivize "gaming the system" by encouraging participants to treat the challenge as we aim to treat the real data LSST will produce. We present our investigation of the optimal choice of the criterion for winning PLAsTiCC in the presence of these unique issues.
- Publication:
-
American Astronomical Society Meeting Abstracts #233
- Pub Date:
- January 2019
- Bibcode:
- 2019AAS...23321205M