Next generation spectro-polarimetric broadband surveys will probe cosmic magnetic fields in unprecedented detail, using the magneto-optical effect known as Faraday rotation. However, non-parametric methods such as RM CLEAN can introduce non-physical linearly polarized flux. This non-physical flux is introduced into the fitted model across negative wavelengths squared and leads to potentially incorrect and non-verifiable Faraday rotation structures that are consistent with the observed data but violate conservation of energy or are difficult to detect. We introduce a novel method called non-parametric $QU$-fitting and use it to show that constraining the non-observable flux in the set of solutions to zero provides a model Faraday depth spectrum that avoids non-physical structures in Faraday depth, a property that is needed when comparing theoretical magnetic field models to fitted Faraday rotation signals. We verify this approach both on simulated and observed broadband data sets, demonstrating that the avoidance of non-physical linear polarized flux in model fitting is a critical requirement in future broadband polarimetric observations and surveys.