Throughout physics, stable composite objects are usually formed by way of attractive forces, which allow the constituents to lower their energy by binding together. Repulsive forces separate particles in free space. However, in a structured environment such as a periodic potential and in the absence of dissipation, stable composite objects can exist even for repulsive interactions. Here we report the observation of such an exotic bound state, which comprises a pair of ultracold rubidium atoms in an optical lattice. Consistent with our theoretical analysis, these repulsively bound pairs exhibit long lifetimes, even under conditions when they collide with one another. Signatures of the pairs are also recognized in the characteristic momentum distribution and through spectroscopic measurements. There is no analogue in traditional condensed matter systems of such repulsively bound pairs, owing to the presence of strong decay channels. Our results exemplify the strong correspondence between the optical lattice physics of ultracold bosonic atoms and the Bose-Hubbard model-a link that is vital for future applications of these systems to the study of strongly correlated condensed matter and to quantum information.