We study the effects of discretization on the U(1) symmetric XY model in two dimensions using the Higher Order Tensor Renormalization Group (HOTRG) approach. Regarding the $Z_N$ symmetric clock models as specific discretizations of the XY model, we compare those discretizations to ones from truncations of the tensor network formulation of the XY model based on a character expansion, and focus on the differences in their phase structure at low temperatures. We also divide the tensor network formulations into core and interaction tensors and show that the core tensor has the dominant influence on the phase structure. Lastly, we examine a perturbed form of the XY model that continuously interpolates between the XY and clock models. We examine the behavior of the additional phase transition caused by the perturbation as the magnitude of perturbation is taken to zero. We find that this additional transition has a non-zero critical temperature as the perturbation vanishes, suggesting that even small perturbations can have a significant effect on the phase structure of the theory.