Gamma-ray bursts (GRBs) come in two classes: long (> 2s), soft-spectrum bursts and short, hard events. Most progress has been made on understanding the long GRBs, which are typically observed at high redshift (z ~ 1) and found in subluminous star-forming host galaxies. They are likely to be produced in core-collapse explosions of massive stars. In contrast, no short GRB had been accurately (< 10'') and rapidly (minutes) located. Here we report the detection of the X-ray afterglow from-and the localization of-the short burst GRB 050509B. Its position on the sky is near a luminous, non-star-forming elliptical galaxy at a redshift of 0.225, which is the location one would expect if the origin of this GRB is through the merger of neutron-star or black-hole binaries. The X-ray afterglow was weak and faded below the detection limit within a few hours; no optical afterglow was detected to stringent limits, explaining the past difficulty in localizing short GRBs.