This paper studies the dynamics of continuous time Markov chains on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in biology. We provide threshold criteria in terms of easily computable parameters for various dynamical properties such as explosivity, recurrence, transience, certain absorption, positive/null recurrence, implosivity, and existence and non-existence of moments of hitting times. In particular, simple sufficient conditions for exponential ergodicity of stationary distributions and quasi-stationary distributions are obtained, and the few gap cases are identified and well-illustrated by examples. Subtle differences in conditions for different dynamical properties are revealed in terms of examples. Finally, we apply our results to stochastic reaction networks, an extended class of branching processes, a general bursty single-cell stochastic gene expression model, and population processes, none of which are birth-death processes.