Nonlinear optical phenomena on rough self-affine metal surfaces are theoretically studied. Placing nonlinearly polarizable molecules on such surfaces results in strong enhancement of optical nonlinearities. A quasistatic approximation is used to calculate local-enhancement factors for the second and third harmonic generation, degenerate four-wave mixing, and nonlinear Kerr effect. The calculations show that the average enhancement factors on a self-affine surface can be as large as 107 and 1015 for optical nonlinearities of the second and third order, respectively, with the maximum average enhancement in the infrared spectral range. Strong spatial inhomogeneity of local-enhancement distribution is demonstrated for the second and third harmonic generation. The local enhancement can exceed the average by several orders of magnitude, reaching extremely high values. Sharp peaks in local-field intensities at fundamental and generated frequencies are localized in spatially separated nanometer-sized areas of the film.