2+1D multi-component $U(1)$ gauge theories with a Chern-Simons (CS) term provide a simple and complete characterization of 2+1D Abelian topological orders. In this paper, we extend the theory by taking the number of component gauge fields to infinity and find that they can describe interesting types of 3+1D "fractonic" order. "Fractonic" describes the peculiar phenomena that point excitations in certain strongly interacting systems either cannot move at all or are only allowed to move in a lower dimensional sub-manifold. In the simplest cases of infinite-component CS gauge theory, different components do not couple to each other and the theory describes a decoupled stack of 2+1D fractional Quantum Hall systems with quasi-particles moving only in 2D planes -- hence a fractonic system. We find that when the component gauge fields do couple through the CS term, more varieties of fractonic orders are possible. For example, they may describe foliated fractonic systems for which increasing the system size requires insertion of nontrivial 2+1D topological states. Moreover, we find examples which lie beyond the foliation framework, characterized by 2D excitations of infinite order and braiding statistics that are not strictly local.