Weakly interacting massive particles (WIMPs) are arguably the most natural dark matter candidates from a particle physics point of view. After their number density has frozen out in the early universe, thereby setting their relic density today, WIMPs are still kept very close to thermal equilibrium by scattering events with standard model particles. The complete decoupling from the thermal bath happens as late as at temperatures of around ~1-10 MeV and provides an important cosmological scale that can directly be translated into a small scale cut-off of the spectrum of matter density fluctuations. We present here a full analytic treatment of the kinetic decoupling process from first principles. This allows an exact determination of the decoupling scale, for arbitrary WIMP candidates and any scattering amplitude. As an application, we consider the situation of the lightest supersymmetric particle as well as the lightest Kaluza Klein particle that arises in theories with universal extra dimensions; furthermore, we show that our formalism can also easily be applied to, e.g., the evolution of the non-relativistic electrons into the recombination regime. Finally, we comment on the impacts for the smallest gravitationally bound structures and the prospects for the indirect detection of dark matter.