  Existing regression models tend to fall short in both accuracy and
uncertainty estimation when the label distribution is imbalanced. In this
paper, we propose a probabilistic deep learning model, dubbed variational
imbalanced regression (VIR), which not only performs well in imbalanced
regression but naturally produces reasonable uncertainty estimation as a
byproduct. Different from typical variational autoencoders assuming I.I.D.
representations (a data point's representation is not directly affected by
other data points), our VIR borrows data with similar regression labels to
compute the latent representation's variational distribution; furthermore,
different from deterministic regression models producing point estimates, VIR
predicts the entire normal-inverse-gamma distributions and modulates the
associated conjugate distributions to impose probabilistic reweighting on the
imbalanced data, thereby providing better uncertainty estimation. Experiments
in several real-world datasets show that our VIR can outperform
state-of-the-art imbalanced regression models in terms of both accuracy and
uncertainty estimation. Code will soon be available at
https://github.com/Wang-ML-Lab/variational-imbalanced-regression.
