In this paper, we propose an efficient, high order accurate and asymptotic-preserving (AP) semi-Lagrangian (SL) method for the BGK model with constant or spatially dependent Knudsen number. The spatial discretization is performed by a mass conservative nodal discontinuous Galerkin (NDG) method, while the temporal discretization of the stiff relaxation term is realized by stiffly accurate diagonally implicit Runge-Kutta (DIRK) methods along characteristics. Extra order conditions are enforced for asymptotic accuracy (AA) property of DIRK methods when they are coupled with a semi-Lagrangian algorithm in solving the BGK model. A local maximum principle preserving (LMPP) limiter is added to control numerical oscillations in the transport step. Thanks to the SL and implicit nature of time discretization, the time stepping constraint is relaxed and it is much larger than that from an Eulerian framework with explicit treatment of the source term. Extensive numerical tests are presented to verify the high order AA, efficiency and shock capturing properties of the proposed schemes.