It has been argued that the dominant non-Newtonian creep mechanisms of water ice make the ice shell above Callisto's ocean, and by inference all radiogenically heated ice I shells in the outer Solar System, stable against solid-state convective overturn. Conductive heat transport and internal melting (oceans) are therefore predicted to be, or have been, widespread among midsize and larger icy satellites and Kuiper Belt objects. Alternatively, at low stresses (where non-Newtonian viscosities can be arbitrarily large), convective instabilities may arise in the diffusional creep regime for arbitrarily small temperature perturbations. For Callisto, ice viscosities are low enough that convection is expected over most of geologic time above the internal liquid layer for plausible ice grain sizes (⩽a few mm); the alternative for early Callisto, a conducting shell over a very deep ocean (>450 km), is not compatible with Callisto's present partially differentiated state. Moreover, if convection is occurring today, the stagnant lid would be quite thick (∼100 km) and compatible with the lack of active geology. Nevertheless, Callisto's steady-state heat flow may have fallen below the convective minimum for its ice I shell late in Solar System history. In this case convection ends, the ice shell melts back at its base, and the internal ocean widens considerably. The presence of such an ocean, of order 200 km thick, is compatible with Callisto's moment-of-inertia, but its formation would have caused an ∼0.25% radial expansion. The tectonic effects of such a late, slow expansion are not observed, so convection likely persists in Callisto, possibly subcritically. Ganymede, due to its greater size, rock fraction and full differentiation, has a substantially higher heat flow than Callisto and has not reached this tectonic end state. Titan, if differentiated, and Triton should be more similar to Ganymede in this regard. Pluto, like Callisto, may be near the tipping point for convective shutdown, but uncertainties in its size and rock fraction prevent a more definitive assessment.