Stars show various amounts of radial velocity (RV) jitter due to varying stellar activity levels. The typical amount of RV jitter as a function of stellar age and observational timescale has not yet been systematically quantified, although it is often larger than the instrumental precision of modern high-resolution spectrographs used for Doppler planet detection and characterization. We aim to empirically determine the intrinsic stellar RV variation for mostly G and K dwarf stars on different timescales and for different stellar ages independently of stellar models. We also focus on young stars ($\lesssim$ 30 Myr), where the RV variation is known to be large. We use archival FEROS and HARPS RV data of stars which were observed at least 30 times spread over at least two years. We then apply the pooled variance (PV) technique to these data sets to identify the periods and amplitudes of underlying, quasiperiodic signals. We show that the PV is a powerful tool to identify quasiperiodic signals in highly irregularly sampled data sets. We derive activity-lag functions for 20 putative single stars, where lag is the timescale on which the stellar jitter is measured. Since the ages of all stars are known, we also use this to formulate an activity--age--lag relation which can be used to predict the expected RV jitter of a star given its age and the timescale to be probed. The maximum RV jitter on timescales of decades decreases from over 500 m/s for 5 Myr-old stars to 2.3 m/s for stars with ages of around 5 Gyr. The decrease in RV jitter when considering a timescale of only 1 d instead of 1 yr is smaller by roughly a factor of 4 for 5 Myr old stars, and a factor of 1.5 for stars with an age of 5 Gyr. The rate at which the RV jitter increases with lag strongly depends on stellar age and ranges from a few days for a few 10 Myr old stars to presumably decades for stars with an age of a few gigayears.