The core accretion theory for giant planet formation predicts enrichment of elemental abundances in planetary envelopes caused by runaway accretion of planetesimals, which is consistent with measured super-solar abundances of C, N, P, S, Xe, and Ar in Jupiter's atmosphere. However, the abundance of O, which is expected to be the most dominant constituent of planetesimals, is unknown for solar system giant planets, owing to the condensation of water in their ultra-cold atmospheres, thereby posing a key unknown in solar system formation. On the other hand, hundreds of extrasolar "hot Jupiters" are known with very high temperatures (>=1000 K), making them excellent targets to measure H2O abundances and, hence, oxygen in their atmospheres. We constrain the atmospheric H2O abundances in three hot Jupiters (HD 189733b, HD 209458b, and WASP-12b), spanning a wide temperature range (1200-2500 K), using their near-infrared transmission spectra obtained using the Wide Field Camera 3 instrument on board the Hubble Space Telescope. We report conclusive measurements of H2O in HD 189733b and HD 209458b, while that in WASP-12b is not well constrained by present data. The data allow nearly solar as well as significantly sub-solar abundances in HD 189733b and WASP-12b. However, for HD 209458b, we report the most precise H2O measurement in an exoplanet to date that suggests a ~20-135 × sub-solar H2O abundance. We discuss the implications of our results on the formation conditions of hot Jupiters and on the likelihood of clouds in their atmospheres. Our results highlight the critical importance of high-precision spectra of hot Jupiters for deriving their H2O abundances.