  Distribution shifts pose significant challenges for model calibration and
model selection tasks in the unsupervised domain adaptation problem -- a
scenario where the goal is to perform well in a distribution shifted domain
without labels. In this work, we tackle difficulties coming from distribution
shifts by developing a novel importance weighted group accuracy estimator.
Specifically, we present a new perspective of addressing the model calibration
and model selection tasks by estimating the group accuracy. Then, we formulate
an optimization problem for finding an importance weight that leads to an
accurate group accuracy estimation with theoretical analyses. Our extensive
experiments show that our approach improves state-of-the-art performances by
22% in the model calibration task and 14% in the model selection task.
