Broadband excitation of plasmons allows control of light-matter interaction with nanometric precision at femtosecond timescales. Research in the field has spiked in the past decade in an effort to turn ultrafast plasmonics into a diagnostic, microscopy, computational, and engineering tool for this novel nanometric-femtosecond regime. Despite great developments, this goal has yet to materialize. Previous work failed to provide the ability to engineer and control the ultrafast response of a plasmonic system at will, needed to fully realize the potential of ultrafast nanophotonics in physical, biological, and chemical applications. Here, we perform systematic measurements of the coherent response of plasmonic nanoantennas at femtosecond timescales and use them as building blocks in ultrafast plasmonic structures. We determine the coherent response of individual nanoantennas to femtosecond excitation. By mixing localized resonances of characterized antennas, we design coupled plasmonic structures to achieve well-defined ultrafast and phase-stable field dynamics in a predetermined nanoscale hotspot. We present two examples of the application of such structures: control of the spectral amplitude and phase of a pulse in the near field, and ultrafast switching of mutually coherent hotspots. This simple, reproducible and scalable approach transforms ultrafast plasmonics into a straightforward tool for use in fields as diverse as room temperature quantum optics, nanoscale solid-state physics, and quantum biology.