We present the near-infrared observation of the zodiacal light with the IRTS (Infrared Telescope in Space), a small cryogenically cooled orbital infrared telescope. The observed spectra of the sky brightness at wavelengths from 1.4 mu m to 4.0 mu m are characterized by a smooth stellar-like spectrum at wavelengths shorter than 3.2 mu m, and enhanced emission at the long wavelengths, which is attributed to thermal emission from interplanetary dust (IPD). The measured sky brightness has a clear dependence on the ecliptic latitude, implying that the zodiacal light is a dominant emission component of the near-infrared sky. The spectrum of the zodiacal light, itself, was obtained by subtracting other emission components. Although it is fairly close to the solar spectrum at wavelengths between 1.8 and 3.2 mu m, it clearly shows a decrement in the scattering efficiency at the short-wavelength end. The zodiacal light at 1.4--3.2 mu m is about twice as bright as that extrapolated from optical measurements. These results indicate that large dust is responsible for the near-infrared zodiacal light; they also suggest that the IPD that has a spectral reflectance similar to the S-type asteroids could be responsible for the near-infrared zodiacal light.