  Distribution shift is an important concern in deep image classification,
produced either by corruption of the source images, or a complete change, with
the solution involving domain adaptation. While the primary goal is to improve
accuracy under distribution shift, an important secondary goal is uncertainty
estimation: evaluating the probability that the prediction of a model is
correct. While improving accuracy is hard, uncertainty estimation turns out to
be frustratingly easy. Prior works have appended uncertainty estimation into
the model and training paradigm in various ways. Instead, we show that we can
estimate uncertainty by simply exposing the original model to corrupted images,
and performing simple statistical calibration on the image outputs. Our
frustratingly easy methods demonstrate superior performance on a wide range of
distribution shifts as well as on unsupervised domain adaptation tasks,
measured through extensive experimentation.
