Quantum multifractality is a fundamental property of systems such as noninteracting disordered systems at an Anderson transition and many-body systems in Hilbert space. Here we discuss the origin of the presence or absence of a fundamental symmetry related to this property. The anomalous multifractal dimension Δq is used to characterize the structure of quantum states in such systems. Although the multifractal symmetry relation Δq=Δ1 −q is universally fulfilled in many known systems, recently some important examples have emerged where it does not hold. We show that this is the result of two different mechanisms. The first one was already known and is related to Gaussian fluctuations well described by random matrix theory. The second one, not previously explored, is related to the presence of an algebraically localized envelope. While the effect of Gaussian fluctuations can be removed by coarse graining, the second mechanism is robust to such a procedure. We illustrate the violation of the symmetry due to algebraic localization on two systems of very different nature, a 1D Floquet critical system and a model corresponding to Anderson localization on random graphs.