Marine calcifying organisms that produce sediments and build reefs generally have skeletons and shells that are composed of either aragonite or calcite. Long-term changes in the estimated Mg/Ca ratios of sea water tend to correspond to changes in the prevailing mineralogy of these creatures. High Mg/Ca ratios are expected to favour the spread of aragonitic organisms, whereas calcitic taxa are thought to benefit from low Mg/Ca ratios. Here we test these patterns throughout the Phanerozoic eon and assess the relative impacts of changing ocean chemistry and mass extinctions on the evolutionary success of calcifying organisms. We find that mass extinctions are more important in regulating long-term patterns of skeletal mineralogy than the Mg/Ca ratios of the global oceans. Furthermore, selective recovery from mass extinctions is usually more important than selective extinction, in driving the Phanerozoic pattern of skeletal mineralogy. But even in the recovery phase there is no clear connection between changes in the dominance of aragonite or calcite and the Mg/Ca ratio of the oceans, thus providing further evidence for the complexity of biotic recoveries.