Transfer Knowledge from Head to Tail: Uncertainty Calibration under Longtailed Distribution
Abstract
How to estimate the uncertainty of a given model is a crucial problem. Current calibration techniques treat different classes equally and thus implicitly assume that the distribution of training data is balanced, but ignore the fact that realworld data often follows a longtailed distribution. In this paper, we explore the problem of calibrating the model trained from a longtailed distribution. Due to the difference between the imbalanced training distribution and balanced test distribution, existing calibration methods such as temperature scaling can not generalize well to this problem. Specific calibration methods for domain adaptation are also not applicable because they rely on unlabeled target domain instances which are not available. Models trained from a longtailed distribution tend to be more overconfident to head classes. To this end, we propose a novel knowledgetransferringbased calibration method by estimating the importance weights for samples of tail classes to realize longtailed calibration. Our method models the distribution of each class as a Gaussian distribution and views the source statistics of head classes as a prior to calibrate the target distributions of tail classes. We adaptively transfer knowledge from head classes to get the target probability density of tail classes. The importance weight is estimated by the ratio of the target probability density over the source probability density. Extensive experiments on CIFAR10LT, MNISTLT, CIFAR100LT, and ImageNetLT datasets demonstrate the effectiveness of our method.
 Publication:

arXiv eprints
 Pub Date:
 April 2023
 DOI:
 10.48550/arXiv.2304.06537
 arXiv:
 arXiv:2304.06537
 Bibcode:
 2023arXiv230406537C
 Keywords:

 Computer Science  Computer Vision and Pattern Recognition