Temporally shaping the density of electron beams using light forms the basis for a wide range of established and emerging technologies, including free-electron lasers and attosecond electron microscopy. The modulation depth of compressed electron pulses is a key figure of merit limiting applications. In this work, we present an approach for generating background-free attosecond electron pulse trains by sequential inelastic electron-light scattering. Harnessing quantum interference in the fractional Talbot effect, we suppress unwanted background density in electron compression by several orders of magnitude. Our results will greatly enhance applications of coherent electron-light scattering, such as stimulated cathodoluminescence and streaking.