Magnetic Sensitivity Beyond the Projection Noise Limit by Spin Squeezing

We report the generation of spin squeezing and entanglement in a magnetically sensitive atomic ensemble, and entanglement-enhanced field measurements with this system. A maximal m_f=±1 Raman coherence is prepared in an ensemble of 8.5×10⁵ laser-cooled Rb87 atoms in the f=1 hyperfine ground state, and the collective spin is squeezed by synthesized optical quantum nondemolition measurement. This prepares a state with large spin alignment and noise below the projection-noise level in a mixed alignment-orientation variable. 3.2 dB of noise reduction is observed and 2.0 dB of squeezing by the Wineland criterion, implying both entanglement and metrological advantage. Enhanced sensitivity is demonstrated in field measurements using alignment-to-orientation conversion.