The non-intrusive generalized Polynomial Chaos (gPC) method is a popular computational approach for solving partial differential equations (PDEs) with random inputs. The main hurdle preventing its efficient direct application for high-dimensional input parameters is that the size of many parametric sampling meshes grows exponentially in the number of inputs (the "curse of dimensionality"). In this paper, we design a weighted version of the reduced basis method (RBM) for use in the non-intrusive gPC framework. We construct an RBM surrogate that can rigorously achieve a user-prescribed error tolerance, and ultimately is used to more efficiently compute a gPC approximation non-intrusively. The algorithm is capable of speeding up traditional non-intrusive gPC methods by orders of magnitude without degrading accuracy, assuming that the solution manifold has low Kolmogorov width. Numerical experiments on our test problems show that the relative efficiency improves as the parametric dimension increases, demonstrating the potential of the method in delaying the curse of dimensionality. Theoretical results as well as numerical evidence justify these findings.