We study collective modes of superfluid Bose gases in optical lattices at commensurate fillings. We focus on the vicinity of the quantum phase transition to the Mott insulator, where there exists the Higgs amplitude mode in addition to the Nambu-Goldstone phase mode associated with the spontaneous U(1) symmetry breaking. We analyze finite-temperature effects on the damping of the collective modes by using an effective spin-1 model and the field theoretical methods based on the finite-temperature Green's function. We calculate the damping rates up to 1-loop order and evaluate them analytically and numerically. We show that the damping rate of the Higgs mode increases with increasing the temperature but it remains underdamped up to a typical temperature achieved in experiments. Moreover, we find that the Nambu-Goldstone mode attenuates via a Landau damping process resulting from interactions with the Higgs mode and it can be overdamped at the typical temperature in a certain parameter region.