The constraints which the observed Z > 4 quasars place on theories of cosmic structure formation are developed. On the basis of quite conservative assumptions (concerning bolometric corrections, cosmological parameters, the speed and efficiency of the accretion flow, and several other factors), it is shown that such theories need to account for a population of >~ 10^8^M_sun_ black holes surrounded by a similar mass of accretion fuel at an overdensity >~ 10^6^. It is further argued that these quasar systems are likely to be embedded in host objects with at least 100 times more baryonic mass at mean densities corresponding to virilization. For astrophysically plausible scenarios the observed >~ 10^46^ ergs/s sources may well be the rare brightest members of a far more extensive population of fainter objects so that a substantial fraction of the universe's baryon content could already be in objects similar to the quasar host objects at Z > 4. Finally, it is argued that these host objects are likely to have formed at redshifts significantly greater than 5 and quite possibly greater than 10 in order to allow sufficient time and/or high enough densities to account for the formation of a massive central black hole. The general conclusion is that the observed Z> 4 quasars suggest that cosmic structure formation was already well advanced at Z ~ 5 when the universe was a small fraction of its present age, about 7% for q_0_ = 1/2 or about 11% for q_0_ = 0.07. The analysis would be significantly complicated and some of the quantitative results substantially modified if either source beaming or gravitational lensing is strongly affecting our observations of the Z> 4 quasar population.