Nonparametric signal separation in very-high-energy gamma ray observations with probabilistic neural networks

An intriguing challenge in observational astronomy is the separation signals in areas where multiple signals intersect. A typical instance of this in very-high-energy (VHE, E$\gtrsim$100 GeV) gamma-ray astronomy is the issue of residual background in observations. This background arises when cosmic-ray protons are mistakenly identified as gamma-rays from sources of interest, thereby blending with signals from astrophysical sources of interest. We introduce a deep ensemble approach to determine a non-parametric estimation of source and background signals in VHE gamma observations, as well as a likelihood-derived epistemic uncertainty on these estimations. We rely on minimal assumptions, exploiting the separability of space and energy components in the signals, and defining a small region in coordinate space where the source signal is assumed to be negligible compared to background signal. The model is applied both on mock observations, including a simple toy case and a realistic simulation of dark matter annihilation in the Galactic center, as well as true observations from the public H.E.S.S. data release, specifically datasets of the Crab nebula and the pulsar wind nebula MSH 15-52. Our method performs well in mock cases, where the ground truth is known, and compares favorably against conventional physical analysis approaches when applied to true observations. In the case of the mock dark matter signal in the Galactic center, our work opens new avenues for component separation in this complex region of the VHE sky.