A novel digital nano-moiré method is proposed to measure the in-plane nanoscopic deformation of an object. In the measurement, the periodic lattice of a single-crystal material acts as a specimen grating while a digital reference grating (DRG) is prepared by computer software. These two gratings overlap to generate a moiré fringe pattern. The preparation of the grating, the formation principle of digital nano-moiré fringes and its relative phase shifting technique are described in detail. A typical experiment was conducted with a highly oriented pyrolytic graphite (HOPG) sample. The residual deformation of the irradiated HOPG sample was measured using this method. The experiment result verifies the feasibility of this method, and demonstrates the potential for further applications.