Next-generation missions designed to detect biosignatures on exoplanets will also be capable of placing constraints on technosignatures (evidence for technological life) on these same worlds. Here, I estimate the detectability of nightside city lights on habitable, Earth-like, exoplanets around nearby stars using direct-imaging observations from the proposed LUVOIR and HabEx observatories, assuming these lights come from high-pressure sodium lamps. I consider how the detectability scales with urbanization fraction: from Earth's value of 0.05%, up to the limiting case of an ecumenopolis -- or planet-wide city. Though an Earth analog would not be detectable by LUVOIR or HabEx, planets around M-dwarfs close to the Sun would show detectable signals at $3\,\sigma$ from city lights, using 300 hours of observing time, for urbanization levels of 0.4% to 3%, while city lights on planets around nearby Sun-like stars would be detectable at urbanization levels of $\gtrsim10\%$. The known planet Proxima b is a particularly compelling target for LUVOIR A observations, which would be able to detect city lights twelve times that of Earth in 300 hours, an urbanization level that is expected to occur on Earth around the mid-22nd-century. An ecumenopolis, or planet-wide city, would be detectable around roughly 30 to 50 nearby stars by both LUVOIR and HabEx, and a survey of these systems would place a $1\,\sigma$ upper limit of $\lesssim2\%$ to $\lesssim4\%$, and a $3\,\sigma$ upper limit $\lesssim10\%$ to $\lesssim15\%$, on the frequency of ecumenopolis planets in the Solar neighborhood assuming no detections.