We have developed a generic physical modeling scheme for high resolution spectroscopy based on simple optical principles. This model predicts the position of centroids for a given set of spectral features with high accuracy. It considers off-plane grating equations and rotations of the different optical elements in order to properly account for tilts in the spectral lines and order curvature. In this way any astronomical spectrograph can be modeled and controlled without the need of commercial ray tracing software. The computations are based on direct ray tracing applying exact corrections to certain surfaces types. This allows us to compute the position on the detector of any spectral feature with high reliability. The parameters of this model, which describe the physical properties of the spectrograph, are continuously optimized to ensure the best possible fit to the observed spectral line positions. We present the physical modeling of CARMENES as a case study. We show that our results are in agreement with commercial ray tracing software. The model prediction matches the observations at a pixel size level, providing an efficient tool in the design, construction and data reduction of high resolution spectrographs.