Contraherent cosheaves are globalizations of cotorsion (or similar) modules over commutative rings obtained by gluing together over a scheme. The category of contraherent cosheaves over a scheme is a Quillen exact category with exact functors of infinite product. Over a quasi-compact semi-separated scheme or a Noetherian scheme of finite Krull dimension (in a different version - over any locally Noetherian scheme), it also has enough projectives. We construct the derived co-contra correspondence, meaning an equivalence between appropriate derived categories of quasi-coherent sheaves and contraherent cosheaves, over a quasi-compact semi-separated scheme and, in a different form, over a Noetherian scheme with a dualizing complex. The former point of view allows us to obtain an explicit construction of Neeman's extraordinary inverse image functor $f^!$ for a morphism of quasi-compact semi-separated schemes $f$. The latter approach provides an expanded version of the covariant Serre-Grothendieck duality theory and leads to Deligne's extraordinary inverse image functor $f^!$ (which we denote by $f^+$) for a morphism of finite type $f$ between Noetherian schemes. Semi-separated Noetherian stacks, affine Noetherian formal schemes, and ind-affine ind-schemes (together with the noncommutative analogues) are briefly discussed in the appendices.