Theoretical investigations of waveguides for FEL operating at submillimeter wavelengths were made. Several types of overmoded waveguides, i.e. rectangular, circular, parallel curved plates waveguide and metal-dielectric waveguide, were studied in order to achieve the best gain/loss ratio in FEL operation. Gain calculations were made in the low gain regime. Waveguide ohmic losses were calculated taking into account the anomalous skin effect and the influence of surface roughness. Numerical calculations made for the parameters of the Israeli tandem electrostatic FEL show that a parallel curved plates waveguide and a metal-dielectric waveguide provide good gain/loss ratios up to frequencies of the order of 1000 GHz (while conventional rectangular and circular waveguides do not).