The non-modal transient growth of perturbations in horizontal and inclined channel flows of two immiscible fluids is studied. 3D perturbations are examined in order to find the optimal perturbations that attain the maximum amplification of perturbation energy at relatively short times. Definition of the energy norm is extended to account for the gravitational potential energy along with the kinetic energy and interfacial capillary energy. Contrarily to the fastest exponential growth, which is reached by essentially 2D perturbations, the maximal non-modal energy growth is attained mostly by three-dimensional spanwise perturbations. Significant transient energy growth is found to occur in linearly stable flow configurations, which, similarly to single phase shear flows, may trigger non-linear destabilizing mechanisms within one of the phases. It is shown that the transient energy growth in linearly stable cases can be accompanied by noticeable interface deformations. Therefore, flow pattern transition due to non-modal transient growth and reduction of the range of operational conditions for which stratified-smooth flow remains stable cannot be ruled out.