  Instance-dependent label noise is realistic but rather challenging, where the
label-corruption process depends on instances directly. It causes a severe
distribution shift between the distributions of training and test data, which
impairs the generalization of trained models. Prior works put great effort into
tackling the issue. Unfortunately, these works always highly rely on strong
assumptions or remain heuristic without theoretical guarantees. In this paper,
to address the distribution shift in learning with instance-dependent label
noise, a dynamic distribution-calibration strategy is adopted. Specifically, we
hypothesize that, before training data are corrupted by label noise, each class
conforms to a multivariate Gaussian distribution at the feature level. Label
noise produces outliers to shift the Gaussian distribution. During training, to
calibrate the shifted distribution, we propose two methods based on the mean
and covariance of multivariate Gaussian distribution respectively. The
mean-based method works in a recursive dimension-reduction manner for robust
mean estimation, which is theoretically guaranteed to train a high-quality
model against label noise. The covariance-based method works in a distribution
disturbance manner, which is experimentally verified to improve the model
robustness. We demonstrate the utility and effectiveness of our methods on
datasets with synthetic label noise and real-world unknown noise.
