Saturn's diffuse E ring consists of many tiny (micron and sub-micron) grains of water ice distributed between the orbits of Mimas and Titan. Various gravitational and non-gravitational forces perturb these particles' orbits, causing the ring's local particle density to vary noticeably with distance from the planet, height above the ring-plane, hour angle and time. Using remote-sensing data obtained by the Cassini spacecraft in 2005 and 2006, we investigate the E-ring's three-dimensional structure during a time when the Sun illuminated the rings from the south at high elevation angles (>15°). These observations show that the ring's vertical thickness grows with distance from Enceladus' orbit and its peak brightness density shifts from south to north of Saturn's equator plane with increasing distance from the planet. These data also reveal a localized depletion in particle density near Saturn's equatorial plane around Enceladus' semi-major axis. Finally, variations are detected in the radial brightness profile and the vertical thickness of the ring as a function of longitude relative to the Sun. Possible physical mechanisms and processes that may be responsible for some of these structures include solar radiation pressure, variations in the ambient plasma, and electromagnetic perturbations associated with Saturn's shadow.