We study the effects of dust grain size on the spectral energy distribution (SED) of spherical circumstellar envelopes. On the basis of the self-similarity relations of dusty SEDs recently derived by Ivezić and Elitzur, we expect an approximate invariance of the IR SED for models with different grain sizes. Approximate invariance follows from the fact that differently sized grains have similar optical properties at long wavelengths at which the dust reprocesses the starlight. In this paper, we discuss the physical requirements on the model parameters to maintain the approximate invariance of the IR SED. Single grain size models are studied for a wide range of grain sizes in three optical depth regimes: optically thin models, moderate opacities, and very optically thick models. In this study, we find limits for the cases in which the IR SED is and is not capable of conveying information about grain sizes and to what extent it does so. We find that approximate invariance occurs for a much larger range of grain sizes than previously believed, and when approximate invariance holds, the SED is controlled mainly by one parameter, the reprocessing optical depth, a quantity that measures the fraction of starlight that is absorbed by the dust grains. Models with a grain size distribution are studied as well. For these models, we find that, in many instances, the concept of approximate invariance may be extended from the IR SED to all wavelengths. This means that, for a wide range of optical depths, models with different grain size distributions produce very similar SEDs, and hence, the reprocessing optical depth is the only quantity that can be unambiguously obtained from the SED. The observational consequences of this result are discussed in detail. Finally, in models with a size distribution, the different grain sizes each have different equilibrium temperatures. The consequences of this effect for the model SED are discussed as well.