The luminosity function, halo masses and stellar masses of luminous Lyman-break galaxies at redshifts 5 < z < 6

We present the results of a study of a large sample of luminous (z'_AB < 26) Lyman-break galaxies (LBGs) in the redshift interval 4.7 < z < 6.3, selected from a contiguous 0.63 deg² area covered by the UKIRT Infrared Deep Sky Survey Ultra Deep Survey and the Subaru XMM-Newton Survey. Utilizing the large area coverage and the excellent available optical+near-infrared data, we use a photometric redshift analysis to derive a new, robust, measurement of the bright end (L >= L^*) of the ultraviolet-selected luminosity function at high redshift. When combined with literature studies of the fainter LBG population, our new sample provides improved constraints on the luminosity function of redshift 5 < z < 6 LBGs over the luminosity range 0.1L^* <~ L <~ 10L^*. A maximum likelihood analysis returns best-fitting Schechter function parameters of M^*₁₅₀₀ = -20.73 +/- 0.11, φ^* = 0.0009 +/- 0.0002Mpc^-3 and α = -1.66 +/- 0.06 for the luminosity function at z = 5, and M^*₁₅₀₀ = -20.04 +/- 0.12, φ^* = 0.0018 +/- 0.0005Mpc^-3 and α = -1.71 +/- 0.11 at z = 6. In addition, an analysis of the angular clustering properties of our LBG sample demonstrates that luminous 5 < z < 6 LBGs are strongly clustered (r₀ = 8.1^+2.1_-1.5h^-1₇₀Mpc), and consistent with the occupation of dark matter haloes with masses of ~=10^11.5-12M_solar. Moreover, by stacking the available multiwavelength imaging data for the high-redshift LBGs, it is possible to place useful constraints on their typical stellar mass. The results of this analysis suggest that luminous LBGs at 5 < z < 6 have an average stellar mass of log₁₀(M/M_solar) = 10.0^+0.2_-0.4, consistent with the results of the clustering analysis assuming plausible values for the ratio of stellar to dark matter. Finally, by combining our luminosity function results with those of the stacking analysis we derive estimates of ~=1 × 10⁷ and ~=4 × 10⁶M_solarMpc^-3 for the stellar mass density at z ~= 5 and 6, respectively.