Let f(t,X) be an irreducible polynomial over the field of rational functions k(t), where k is a number field. Let O be the ring of integers of k. Hilbert's irreducibility theorem gives infinitely many integral specializations of t to values a in O such that f(a,X) is still irreducible. In this paper we study the set Red(O) of those a in O with f(a,X) reducible. We show that Red(O) is a finite set under rather weak assumptions. In particular, several results of K. Langmann, obtained by Diophantine approximation techniques, appear as special cases of some of our results. Our method is completely different. We use elementary group theory, valuation theory, and Siegel's theorem about integral points on algebraic curves. Indeed, using the Siegel-Lang extension of Siegel's theorem, most of our results hold over more general fields.