This paper is the first in a series that will look at the theory of phase transitions from the perspectives of physics and the philosophy of science. The series will consider a group of related concepts derived from condensed matter and statistical physics. The key technical ideas go under the names of "singularity", "order parameter", "mean field theory", "variational method", "correlation length", "universality class", "scale changes", and "renormalization". The first four of these will be considered here. In a less technical vein, the question here is how can matter, ordinary matter, support a diversity of forms. We see this diversity each time we observe ice in contact with liquid water or see water vapor (steam) come up from a pot of heated water. Different phases can be qualitatively different in that walking on ice is well within human capacity, but walking on liquid water is proverbially forbidden to ordinary humans. These differences have been apparent to humankind for millennia, but only brought within the domain of scientific understanding since the 1880s. A phase transition is a change from one behavior to another. A first order phase transition involves a discontinuous jump in some statistical variable. The discontinuous property is called the order parameter. Each phase transition has its own order parameter. The possible order parameters range over a tremendous variety of physical properties. These properties include the density of a liquid-gas transition, the magnetization in a ferromagnet, the size of a connected cluster in a percolation transition, and a condensate wave function in a superfluid or superconductor. A continuous transition occurs when the discontinuity in the jump approaches zero. This article is about statistical mechanics and the development of mean field theory as a basis for a partial understanding of phase transition phenomena. Much of the material in this review was first prepared for the Royal Netherlands Academy of Arts and Sciences in 2006. It has appeared in draft form on the authors' web site (http://jfi.uchicago.edu/~leop/) since then. The title of this article is a hommage to Philip Anderson and his essay "More is Different" (Sci. New Ser. 177(4047):393-396, 1972; N.-P. Ong and R. Bhatt (eds.) More is Different: Fifty Years of Condensed Matter Physics, Princeton Series in Physics, Princeton University Press, 2001) which describes how new concepts, not applicable in ordinary classical or quantum mechanics, can arise from the consideration of aggregates of large numbers of particles. Since phase transitions only occur in systems with an infinite number of degrees of freedom, such transitions are a prime example of Anderson's thesis.