The elliptical orbits resulting from Newtonian gravitation are generated with a multifaceted symmetry, mainly resulting from their conservation of both angular momentum and a vector fixing their orientation in space—the Laplace or Runge-Lenz vector. From the ancient formalisms of celestial mechanics, I show a rather counterintuitive behavior of the classical hydrogen atom, whose orbits respond in a direction perpendicular to a weak externally-applied electric field. I then show how the same results can be obtained more easily and directly from the intrinsic symmetry of the Kepler problem. If the atom is subjected to an oscillating electric field, it enjoys symmetry in the time domain as well, which is manifest by quasi-energy states defined only modulo ħω. Using the Runge-Lenz vector in place of the radius vector leads to an exactly-solvable model Hamiltonian for an atom in an oscillating electric field—embodying one of the few meaningful exact solutions in quantum mechanics, and a member of an even more exclusive set of exact solutions having a time-dependent Hamiltonian. I further show that, as long as the atom suffers no change in principal quantum number, incident radiation will produce harmonic radiation with polarization perpendicular to the incident radiation. This unusual polarization results from the perpendicular response of the wavefunction, and is distinguished from most usual harmonic radiation resulting from a scalar nonlinear susceptibility. Finally, I speculate on how this radiation might be observed.