The antiferromagnetic critical point of the Potts model on the square lattice was identified by Baxter [R.J. Baxter, Proc. R. Soc. London A 383 (1982) 43] as a staggered integrable six-vertex model. In this work, we investigate the integrable structure of this model. It enables us to derive some new properties, such as the Hamiltonian limit of the model, an equivalent vertex model, and the structure resulting from the Z symmetry. Using this material, we discuss the low-energy spectrum, and relate it to geometrical excitations. We also compute the critical exponents by solving the Bethe equations for a large lattice width N. The results confirm that the low-energy spectrum is a collection of continua with typical exponent gaps of order (.