The purpose of this paper is to study the $L^2$ boundedness of operators of the form \[ f\mapsto \psi(x) \int f(\gamma_t(x)) K(t) dt, \] where $\gamma_t(x)$ is a $C^\infty$ function defined on a neighborhood of the origin in $(t,x)\in \R^N\times \R^n$, satisfying $\gamma_0(x)\equiv x$, $\psi$ is a $C^\infty$ cutoff function supported on a small neighborhood of $0\in \R^n$, and $K$ is a "multi-parameter singular kernel" supported on a small neighborhood of $0\in \R^N$. The goal is, given an appropriate class of kernels $K$, to give conditions on $\gamma$ such that every operator of the above form is bounded on $L^2$. The case when $K$ is a Calderón-Zygmund kernel was studied by Christ, Nagel, Stein, and Wainger; we generalize their conditions to the case when $K$ has a "multi-parameter" structure. For example, when $K$ is given by a "product kernel." Even when $K$ is a Calderón-Zygmund kernel, our methods yield some new results. This is the first paper in a three part series, the later two of which are joint with E. M. Stein. The second paper deals with the related question of $L^p$ boundedness, while the third paper deals with the special case when $\gamma$ is real analytic.