Typically higher-derivative theories are unstable. Instabilities manifest themselves from extra propagating degrees of freedom, which are unphysical. In this paper, we will investigate an infinite derivative field theory and study its true dynamical degrees of freedom via Hamiltonian analysis. In particular, we will show that if the infinite derivatives can be captured by a Gaussian kinetic term, i.e. exponential of entire function, then it is possible to prove that there are only finite number of dynamical degrees of freedom. We will further extend our investigation into infinite derivative theory of gravity, and in particular concentrate on ghost free and singularity free theory of gravity, which has been studied extensively in the Lagrangian approach. Here we will show from the Hamiltonian perspective that there are only finite number of degrees of freedom.