Whispering gallery modes were used for very accurate permittivity and dielectric loss measurements of ultralow loss isotropic and uniaxially anisotropic single crystals. Several materials including sapphire, YAG, quartz, and SrLaAlO4 were measured. The total absolute uncertainty in the real part of permittivity tensor components was estimated to be ±0.1%, limited principally by the uncertainty in sample dimensions. Imaginary parts of permittivities were measured with uncertainties of about 10%, limited by the accuracy of Q-factor measurements of whispering gallery modes. It has been observed that, for most crystals, dielectric losses can be approximated by a power function of absolute temperature only in limited temperature ranges. At temperatures between 4-50 K, losses are often affected by impurities, which are always present in real crystals.