The study of the space density of bright active galactic nuclei (AGNs) at z > 4 has been subject to extensive effort given its importance in the estimation of cosmological ionizing emissivity and growth of supermassive black holes. In this context we have recently derived high space densities of AGNs at z ∼ 4 and -25 < M 1450 < -23 in the Cosmic Evolution Survey (COSMOS) field from a spectroscopically complete sample. In the present paper we attempt to extend the knowledge of the AGN space density at fainter magnitudes (-22.5 < M 1450 < -18.5) in the 4 < z < 6.1 redshift interval by means of a multiwavelength sample of galaxies in the Cosmic Assembly Near-IR Deep Extragalactic Legacy Survey (CANDELS) GOODS-South, GOODS-North, and EGS fields. We use an updated criterion to extract faint AGNs from a population of near-IR (rest-frame UV) selected galaxies at photometric z > 4 showing X-ray detection in deep Chandra images available for the three CANDELS fields. We have collected a photometric sample of 32 AGN candidates in the selected redshift interval, six of which having spectroscopic redshifts. Including our COSMOS sample as well as other bright QSO samples allows a first guess on the shape of the UV luminosity function (LF) at z ∼ 4.5. The resulting emissivity and photoionization rate appear consistent with that derived from the photoionization level of the intergalactic medium at z ∼ 4.5. An extrapolation to z ∼ 5.6 suggests an important AGN contribution to the ionization of intergalactic medium if there are no significant changes in the shape of the UV LF.