The universal enveloping algebra of any simple Lie algebra g contains a family of commutative subalgebras, called the quantum shift of argument subalgebras math.RT/0606380, math.QA/0612798. We prove that generically their action on finite-dimensional modules is diagonalizable and their joint spectra are in bijection with the set of monodromy-free opers for the Langlands dual group of G on the projective line with regular singularity at one point and irregular singularity of order two at another point. We also prove a multi-point generalization of this result, describing the spectra of commuting Hamiltonians in Gaudin models with irregular singulairity. In addition, we show that the quantum shift of argument subalgebra corresponding to a regular nilpotent element of g has a cyclic vector in any irreducible finite-dimensional g-module. As a byproduct, we obtain the structure of a Gorenstein ring on any such module. This fact may have geometric significance related to the intersection cohomology of Schubert varieties in the affine Grassmannian.