The application of pressure to materials can reveal unexpected chemistry. Under compression, noble gases form stoichiometric van der Waals (vdW) compounds with closed-shell molecules such as hydrogen, leading to a variety of unusual structures. We have synthesised Kr(H2)4 for the first time in a diamond-anvil high-pressure cell at pressures >=5.3 GPa and characterised its structural and vibrational properties to above 50 GPa. The structure of Kr(H2)4, as solved by single-crystal synchrotron X-ray diffraction, is face-centred cubic (fcc) with krypton atoms forming isolated octahedra at fcc sites. Rotationally disordered H2 molecules occupy four different, interstitial sites, consistent with the observation of four Raman active H2 vibrons. The discovery of Kr(H2)4 expands the range of pressure-stabilised, hydrogen-rich vdW solids, and, in comparison with the two known rare-gas-H2 compounds, Xe(H2)8 and Ar(H2)2, reveals an increasing change in hydrogen molecular packing with increasing rare gas atomic number.