A new regularization and renormalization procedure is presented. It is particularly well suited for the treatment of gauge theories. The method works for theories that were known to be renormalizable as well as for Yang-Mills type theories. Overlapping divergencies are disentangled. The procedure respects unitarity, causality and allows shifts of integration variables. In non-anomalous cases also Ward identities are satisfied at all stages. It is transparent when anomalies, such as the Bell-Jackiw-Adler anomaly, may occur.