How and when do galaxies form? Studies of the microwave background radiation reveal that the universe is spectacularly homogenous at redshift z~1000. Observations of the local universe reveal that by z=0 much of the luminous matter has condensed into mature, gravitationally bound structures. One of the primary challenges to astronomers today is to achieve a robust understanding of this process of galaxy formation and evolution. Locating and studying young galaxies at large look-back times is an essential aspect of this program. We review the systematic observational techniques used to identify galaxies at early cosmic epochs. In the past few years, the study of normal, star-forming galaxies at z>3 has become possible; indeed, successful methods have been developed to push the frontier past z=5. We are now directly observing individual galaxies within a Gyr of the big bang. We present a detailed review of the many search methods used for identifying distant galaxies, consider the biases inherent in different search strategies, and discuss early results of these studies. We conclude with goals for future studies at the start of the 21st century.