We describe an all-electron $G_0W_0$ implementation for periodic systems with $k$-point sampling implemented in a crystalline Gaussian basis. Our full-frequency $G_0W_0$ method relies on efficient Gaussian density fitting integrals and includes both analytic continuation and contour deformation schemes. Due to the compactness of Gaussian bases, no virtual state truncation is required as is seen in many plane-wave formulations. Finite size corrections are included by taking the $q \to 0$ limit of the Coulomb divergence. Using our implementation, we study quasiparticle energies and band structures across a range of systems including molecules, semiconductors, rare gas solids, and metals. We find that the $G_0W_0$ band gaps of traditional semiconductors converge rapidly with respect to the basis size, even for the conventionally challenging case of ZnO. Using correlation-consistent bases of polarized triple-zeta quality, we find the mean absolute relative error of the extrapolated $G_0W_0$@PBE band gaps to be only 5.2% when compared to experimental values. For core excitation binding energies (CEBEs), we find that $G_0W_0$ predictions improve significantly over those from DFT if the $G_0W_0$ calculations are started from hybrid functionals with a high percentage of exact exchange.