We describe and demonstrate a method for the computation of quantum dynamics on small, noisy universal quantum computers. This method relies on the idea of `restarting' the dynamics; at least one approximate time step is taken on the quantum computer and then a parameterized quantum circuit ansatz is optimized to produce a state that well approximates the time-stepped results. The simulation is then restarted from the optimized state. By encoding knowledge of the form of the solution in the ansatz, such as ensuring that the ansatz has the appropriate symmetries of the Hamiltonian, the optimized ansatz can recover from the effects of decoherence. This allows for the quantum dynamics to proceed far beyond the standard gate depth limits of the underlying hardware, albeit incurring some error from the optimization, the quality of the ansatz, and the typical time step error. We demonstrate this methods on the Aubry-André model with interactions at half-filling, which shows interesting many-body localization effects in the long time limit. Our method is capable of performing high-fidelity Hamiltonian simulation hundred of time steps longer than the standard Trotter approach. These results demonstrate a path towards using small, lossy devices to calculate quantum dynamics.