Chiral nematic and fluctuation-induced first-order phase transitions in AB-stacked kagome bilayers

We study a Heisenberg-Dzyaloshinskĭ-Moriya Hamiltonian on AB-stacked kagome bilayers at finite temperature. In a large portion of the parameter space, we observe three qualitative changes upon cooling the system: a crossover from a Heisenberg paramagnet to an XY chiral paramagnet, a Kosterlitz-Thouless transition to a chiral nematic phase, and a fluctuation-induced first-order transition to an Ising-like phase. We characterize the properties of phases numerically using Monte Carlo finite-size analysis. To further explain the nature of the observed phase transitions, we develop an analytical coarse-graining procedure that maps the Hamiltonian onto a generalized XY model on a triangular lattice. To leading order, this effective model includes both bilinear and biquadratic interactions and is able to correctly predict the two phase transitions. Lastly, we study the Ising fluctuations at low temperatures and establish that the origin of the first-order transition stems from the quasidegenerate ring manifold in the momentum space.