Optical follow-up of the neutron star-black hole mergers S200105ae and S200115j

LIGO and Virgo's third observing run revealed the first neutron star-black hole (NSBH) merger candidates in gravitational waves. These events are predicted to synthesize r-process elements^1,2 creating optical/near-infrared `kilonova' emission. The joint gravitational wave and electromagnetic detection of an NSBH merger could be used to constrain the equation of state of dense nuclear matter³, and independently measure the local expansion rate of the Universe⁴. Here, we present the optical follow-up and analysis of two of the only three high-significance NSBH merger candidates detected to date, S200105ae and S200115j, with the Zwicky Transient Facility⁵. The Zwicky Transient Facility observed ~48% of S200105ae and ~22% of S200115j's localization probabilities, with observations sensitive to kilonovae brighter than -17.5 mag fading at 0.5 mag d^-1 in the g- and r-bands; extensive searches and systematic follow-up of candidates did not yield a viable counterpart. We present state-of-the-art kilonova models tailored to NSBH systems that place constraints on the ejecta properties of these NSBH mergers. We show that with observed depths of apparent magnitude ~22 mag, attainable in metre-class, wide-field-of-view survey instruments, strong constraints on ejecta mass are possible, with the potential to rule out low mass ratios, high black hole spins and large neutron star radii.