We report the detection with the Proportional Counter Array (PCA) on board the Rossi X-Ray Timing Explorer (RXTE) of millisecond variability in the X-ray emission from the low-mass X-ray binary 4U 1728-34. Pulsations at 363 Hz with amplitudes (rms) of 2.5%--10% are present in six of the eight bursts analyzed to date. The strongest were seen in two successive bursts recorded on 1996 February 16 when the quiescent count rate was near the highest seen by PCA. The pulsations during these bursts show frequency changes of 1.5 Hz during the first few seconds but become effectively coherent as the burst decays. We interpret the 363 Hz pulsations as rotationally induced modulations of inhomogeneous burst emission. This represents the first compelling evidence for a millisecond spin period in a low-mass X-ray binary. Complex, intensity-dependent, millisecond X-ray variability is also present in all the quiescent flux intervals we examined. Most interesting was the behavior as the count rate approached its highest observed level. Two quasi-periodic oscillations (QPOs) were simultaneously observed in the 650--1100 Hz range. Both QPOs increased in frequency together, maintaining a nearly constant frequency separation of about 363 Hz, the spin period inferred from the burst oscillations. This phenomenology is strongly suggestive of the magnetospheric beat frequency model proposed for the horizontal-branch oscillations (HBOs) seen in Z sources. We discuss this and several other possible physical interpretations for the observed X-ray variability.