Little Higgs models offer a new way to address the hierarchy problem, and give rise to a weakly-coupled Higgs sector. These theories predict the existence of new states which are necessary to cancel the quadratic divergences of the Standard Model. The simplest version of these models, the Littlest Higgs, is based on an SU(5)/SO(5) non-linear sigma model and predicts that four new gauge bosons, a weak isosinglet quark, t', with Q = 2/3, as well as an isotriplet scalar field exist at the TeV scale. We consider the contributions of these new states to precision electroweak observables, and examine their production at the Tevatron. We thoroughly explore the parameter space of this model and find that small regions are allowed by the precision data where the model parameters take on their natural values. These regions are, however, excluded by the Tevatron data. Combined, the direct and indirect effects of these new states constrain the `decay constant' f>~3.5 TeV and mt'>~10 TeV. These bounds imply that significant fine-tuning be present in order for this model to resolve the hierarchy problem.