Higher derivative couplings of hypermultiplets to $6D, N=(1,0)$ supergravity are obtained from dimensional reduction of 10D heterotic supergravity that includes order $\alpha'$ higher derivative corrections. Reduction on $T^4$ is followed by a consistent truncation. In the resulting action the hyperscalar fields parametrize the coset $SO(4,4)/(SO(4)\times SO(4))$. While the $SO(4,4)$ symmetry is ensured by Sen's construction based on string field theory, its emergence at the field theory level is a nontrivial phenomenon. A number of field redefinitions in the hypermultiplet sector are required to remove several terms that break the $SO(4)\times SO(4)$ down to its $SO(4)$ diagonal subgroup in the action and the supersymmetry transformation rules. Working with the Lorentz Chern-Simons term modified 3-form field strength, where the spin connection has the 3-form field strength as torsion, is shown to simplify considerably the dimensional reduction.