High-resolution VLA and Ryle Telescope observations, at 0.33,1.5,8.4 and 15 GHz are presented of a complete sample of 20 radio sources that are variable at 151 MHz. In 18 of the sources, the emission is dominated by structure on a scale <= 0.1 arcsec at 8.4 GHz; the spectra of these compact components indicate that structure on this scale probably also dominates the low-frequency emission. Most of the sources belong to the compact steep-spectrum class. Four are identified with quasars with redshifts greater than 1. The remainder are unidentified or identified with a stellar object; it is argued that these are also likely to be at redshifts in excess of 1. The most extreme low-frequency variability is shown by the unresolved sources with steep high-frequency and flat low- frequency spectra, suggesting that these objects have the most compact components at low frequencies.