Quantum liquids of the S =3/2 Kitaev honeycomb and related Kugel-Khomskii models

The S =3/2 Kitaev honeycomb model (KHM) is unique among the spin-S Kitaev models due to a massive ground-state quasidegeneracy that hampered previous numerical and analytical studies. In a recent work [Jin et al., Nat. Commun. 13, 3813 (2022), 10.1038/s41467-022-31503-0], we showed how an SO(6) Majorana parton mean-field theory of the S =3/2 isotropic KHM explains the anomalous features of this Kitaev spin liquid (KSL) in terms of an emergent low-energy Majorana flat band. Away from the isotropic limit, the S =3/2 KSL generally displays a quadrupolar order with gapped or gapless Majorana excitations, features that were quantitatively confirmed by density-matrix renormalization group simulations. In this paper, we explore the connection between the S =3/2 KHM with Kugel-Khomskii models and discover exactly soluble examples for the latter. We perform a symmetry analysis for the variational parton mean-field Ansätze in the spin and orbital basis for different quantum liquid phases of the S =3/2 KHM. Finally, we investigate a proposed time-reversal symmetry-breaking spin liquid induced by [111] single-ion anisotropy and elucidate its topological properties as well as experimental signatures, e.g., an unquantized thermal Hall response.