Spontaneous magnon decay is a generic feature of the magnetic excitations of anisotropic magnets and isotropic magnets with noncollinear order. In this Rapid Communication, we argue that the effect of interactions on one-magnon states can, under many circumstances, be treated in terms of an effective, energy-independent, non-Hermitian Hamiltonian for the magnons. In the vicinity of Dirac or Weyl touching points, we show that the spectral function has a characteristic anisotropy arising from topologically protected exceptional points or lines in the non-Hermitian spectrum. Such features can, in principle, be detected using inelastic neutron scattering or other spectroscopic probes. We illustrate this physics through a concrete example: a honeycomb ferromagnet with Dzyaloshinskii-Moriya exchange. We perform interacting spin-wave calculations of the structure factor and spectral function of this model, showing good agreement with results from a simple effective non-Hermitian model for the splitting of the Dirac point. Finally, we argue that the zoo of known topological protected magnon band structures may serve as a nearly ideal platform for realizing and exploring non-Hermitian physics in solid-state systems.