It is now generally believed that galaxies were built up through gravitational amplification of primordial fluctuations and the subsequent merging of smaller precursor structures. The stars of the structures that assembled to form the Milky Way should make up much or all of its bulge and halo, in which case one hopes to find `fossil' evidence for those precursor structures in the present distribution of halo stars. Confirmation that this process is continuing came with the discovery of the Saggittarius dwarf galaxy, which is being disrupted by the Milky Way, but direct evidence that this process provided the bulk of the Milky Way's population of old stars has hitherto been lacking. Here we show that about ten per cent of the metal-poor stars in the halo of the Milky Way, outside the radius of the Sun's orbit, come from a single coherent structure that was disrupted during or soon after the Galaxy's formation. This object had a highly inclined orbit about the Milky Way at a maximum distance of ~16kpc, and it probably resembled the Fornax and Sagittarius dwarf spheroidal galaxies.