Numerical computations have become a pillar of all modern quantitative sciences. Any computation involves modeling—even if often this step is not made explicit—and any model has to neglect details while still being physically accurate. Equilibrium statistical mechanics guides both the development of models and numerical methods for dynamics obeying detailed balance. For systems driven away from thermal equilibrium, such a universal theoretical framework is missing. For a restricted class of driven systems governed by Markov dynamics and local detailed balance, stochastic thermodynamics has evolved to fill this gap and to provide fundamental constraints and guiding principles. The next step is to advance stochastic thermodynamics from simple model systems to complex systems with tens of thousands or even millions of degrees of freedom. Biomolecules operating in the presence of chemical gradients and mechanical forces are a prime example for this challenge. In this Perspective, we give an introduction to isothermal stochastic thermodynamics geared toward the systematic multiscale modeling of the conformational dynamics of biomolecular and synthetic machines, and we outline some of the open challenges.