This paper investigates the possible effect of grain size distribution on the rheological behavior of polycrystalline materials, specifically on the stress exponent n, activation energy Q, and grain size exponent p in the steady-state deformation. All grains in a specimen are assumed to deform simultaneously by power-law creep and grain boundary diffusion (Coble) creep. It is shown that the overall values of n, Q and p are affected by the sizes of grains and their volume fractions in the specimen. In the case that grains of different sizes deform at the same stress, but at different rates, a small number of fine grains in a coarse polycrystal may initiate an overall Newtonian creep behavior. In the case that grains of different sizes deform at the same strain rate, but support different stresses, a small number of coarse grains in a fine polycrystal may induce an overall power-law creep behavior. When a diverse grain size distribution exists, but only the average grain size is considered, the conclusion about the dominant mechanism operating is misleading. Care should be taken when inferring the overall deformational behavior of rocks containing grains of widely differing sizes based on the microstructures preserved.