We present 10 m Keck spectroscopy and photometry for a sample of 61 small (r1/2 <= 0.5"), faint (I814 <= 23.74), high-surface brightness (μI814 < 22.2 mag arcsec-2) galaxies in fields flanking the Hubble Deep Field. The majority of this empirically defined sample of compact galaxies lies at redshifts 0.4 <~ z <~ 1 (88% completeness in redshift identifications), ruling out a large component of low-redshift galaxies. The number of such galaxies in the range 1.4 <~ z <~ 2.2 is also constrained to <~10%. The majority of the observed galaxies are emission-line systems, while a significant fraction (23%-34%) appear to be normal ellipticals or otherwise early-type systems. One object is an active galactic nucleus, and two are at high redshift (z > 2). The Keck redshift and photometric data are combined with Hubble Space Telescope images to derive luminosities and physical sizes. We also use emission-line widths, where available, to estimate masses. About two-thirds of the emission-line galaxies, or roughly one-half the sample, are small, low-mass, relatively luminous systems with properties resembling those of local H II galaxies. We compare the properties and numbers of these galaxies to the ``bursting dwarf'' model of Babul & Ferguson. Our sample includes many galaxies similar to the model galaxies in the redshift range 0.4 <~ z <~ 0.7, but the majority of our compact galaxies are more luminous (by up to an order of magnitude) than those of the model. The number of galaxies fitting the model parameters are lower by a factor of 2-3 than predicted. An examination of samples used in analyses of disk surface brightness at redshifts z > 0.5 shows that compact galaxies are likely to contribute to the strong disk luminosity evolution found in some studies. Estimates of comoving volume densities indicate that the population of apparent H II galaxies evolves rapidly from redshifts of z ~ 1 to the present. It appears that not all of these galaxies can be progenitors of present-day spheroidal galaxies, although the numbers of them with sizes and masses comparable to spheroidals is not dissimilar to estimates of the local field spheroidal density. We also present 51 additional redshifts, acquired for other projects during the same observing period, for a total of 105 objects with identified redshifts z < 2 in the Hubble Deep Field and its flanking fields.Based in part on observations with the NASA/ESA Hubble Space Telescope, obtained at the Space Telescope Science Institute, which is operated by AURA, Inc., under NASA contract NAS 5-26555. Lick Observatory Bulletin 1361.