To explain experimentally observed oscillatory dynamics of highly efficient process of excitation energy transfer (EET) in Fenna-Matthews-Olson (FMO) complex, most theoretical models assume the same local protein environment around all the bacteriochlorophyll-a (BChla) sites, contradictory to the structural analysis of FMO complex. Using different values of pigment-protein couplings for different BChla sites, measured in the adiabatic limit of electron transfer, we theoretically investigate the effect of inhomogeneous local protein environment on excitation energy transfer. By employing non-Markovian master equation we demonstrate that the asymmetric system-bath coupling leads to the results consistent with the experimental observations. Quantum dynamical simulation suggests that the correlated fuctuations preserve the oscillation of excitation for long time-scales. Further different BChla sites have asymmetric time-scales of oscillations of excitation due to in-homogeneous pigment-protein couplings also.