We report the discovery of low frequency quasi-periodic oscillations centered at 0.11 Hz in the newly discovered 221 s X-ray pulsar XTE J1858+034. Among about 30 known transient X-ray pulsars this is the sixth source in which QPOs have been observed. If the QPOs are produced because of inhomogeneities in the accretion disk at the magnetospheric boundary, the low frequency of the QPOs indicate a large magnetosphere for this pulsar. Both the Keplerian frequency model and the beat frequency model are applicable for production of QPOs in this source. The QPOs and regular pulsations are found to be stronger at higher energy which favours the beat frequency model. The magnetic field of the pulsar is calculated as a function of its distance. The energy spectrum is found to be very hard, consisting of two components, a cut-off power law and an iron fluorescence line.