The evolution of self-gravitating systems, and long-range interacting systems more generally, from initial configurations far from dynamical equilibrium is often described as a simple two-phase process: a first phase of violent relaxation bringing it to a quasistationary state in a few dynamical times, followed by a slow adiabatic evolution driven by collisional processes. In this context the complex spatial structure evident, for example, in spiral galaxies is understood either in terms of instabilities of quasistationary states or as a result of dissipative nongravitational interactions. We illustrate here, using numerical simulations, that purely self-gravitating systems evolving from quite simple initial configurations can in fact give rise easily to structures of this kind, of which the lifetime can be large compared to the dynamical characteristic time but short compared to the collisional relaxation timescale. More specifically, for a broad range of nonspherical and nonuniform rotating initial conditions, gravitational relaxation gives rise quite generically to long-lived nonstationary structures of a rich variety, characterized by spiral-like arms, bars, and even ringlike structures in special cases. These structures are a feature of the intrinsically out-of-equilibrium nature of the system's collapse, associated with a part of the system's mass while the bulk is well virialized. They are characterized by predominantly radial motions in their outermost parts, but also incorporate an extended flattened region which rotates coherently about a well-virialized core of triaxial shape with an approximately isotropic velocity dispersion. We characterize the kinematical and dynamical properties of these complex velocity fields and we briefly discuss the possible relevance of these simple toy models to the observed structure of real galaxies, emphasizing the difference between dissipative and dissipationless disk formation.