For the purpose of understanding second-order scalar PDEs and their hydrodynamic integrability, we introduce G-structures that are induced on hypersurfaces of the space of symmetric matrices (interpreted as the fiber of second-order jet space) and are defined by non-degenerate scalar second-order-only (Hessian) PDEs in any number of variables. The fiber group is a conformal orthogonal group that acts on the space of independent variables, and it is a subgroup of the conformal orthogonal group for a semi-Riemannian metric that exists on the PDE. These G-structures are automatically compatible with the definition of hydrodynamic integrability, so they allow contact-invariant analysis of integrability via moving frames and the Cartan--Kaehler theorem. They directly generalize the GL(2)-structures that arise in the case of Hessian hyperbolic equations in three variables as well as several related geometries that appear in the literature on hydrodynamic integrability. Part I primarily discusses the motivation, the definition, and the solution to the equivalence problem, and Part II will discuss integrability in detail.