We present new radio observations of the binary neutron star merger GW170817 carried out with the Karl G. Jansky Very large Array (VLA) more than $3\,$yrs after the merger. Our combined dataset is derived by co-adding more than $\approx32\,$hours of VLA time on-source, and as such provides the deepest combined observation (RMS sensitivity $\approx 0.99\,\mu$Jy) of the GW170817 field obtained to date at $3\,$GHz. We find no evidence for a late-time radio re-brightening at a mean epoch of $t\approx 1200\,$d since merger, in contrast to a $2-3\,\sigma$ excess observed at X-ray wavelengths at the same mean epoch. Our measurements agree with expectations from the post-peak decay of the radio afterglow of the GW170817 structured jet. Using these results, we constrain the parameter space of models that predict a late-time radio re-brightening possibly arising from the high-velocity tail of the GW170817 kilonova ejecta, which would dominate the radio and X-ray emission years after the merger (once the structured jet afterglow fades below detection level). Our results point to a steep energy-speed distribution of the kilonova ejecta (with energy-velocity power law index $\alpha \gtrsim 5$). We suggest possible implications of our radio analysis, when combined with the recent tentative evidence for a late-time re-brightening in the X-rays, and highlight the need for continued radio-to-X-ray monitoring to test different scenarios.