The breakdown of ergodic behaviour is discussed as a general phenomenon in condensed matter physics. Broken symmetry is a particular case of this broken ergodicity. In a system that is non-ergodic on physical timescales the phase point is effectively confined in one subregion or component of phase space. Theoretical treatments of such systems should compute thermal averages over one component at a time. The probability distribution of physical properties can then be obtained from occurrence probabilities for different components, and moments of these distributions may be used for predicting the results of typical measurements. A two-level statistical mechanics is therefore proposed. Various aspects of the breakdown of ergodicity are discussed, including: the definition of components; mechanisms for the confinement of components; methods for computing the properties of one component; the choice of occurrence probabilities for different components; and average thermodynamic properties of broken ergodicity systems. The theory is illustrated by application to well-understood systems. It is also applied to the spin glass, which is reviewed in some detail. The broken ergodicity viewpoint is combined with a suggested characterization of the components in a spin glass, involving a bifurcation cascade. Together these provide a qualitative explanation for the irreversibility signature, the long time decays, the apparent failure of linear response theory and Maxwell relations, and blocking.