Motivated by two recent experiments in which thermal properties of complex many-body systems were successfully reproduced on a commercially available quantum annealer, we examine the extent to which quantum annealing hardware can reliably sample from the thermal state associated with a target quantum Hamiltonian. We address this question by studying the thermal properties of the canonical one-dimensional transverse-field Ising model on a D-Wave 2000Q quantum annealing processor. We find that the quantum processor fails to produce the correct expectation values predicted by Quantum Monte Carlo. Comparing to master equation simulations, we find that this discrepancy is best explained by how the measurements at finite transverse fields are enacted on the device. Specifically, measurements at finite transverse field require the system to be quenched from the target Hamiltonian to a Hamiltonian with negligible transverse field, and this quench is too slow. We elaborate on how the limitations imposed by such hardware make it an unlikely candidate for studying the thermal properties of generic quantum many-body systems.