  In this work, we highlight and perform a comprehensive study on calibration
attacks, a form of adversarial attacks that aim to trap victim models to be
heavily miscalibrated without altering their predicted labels, hence
endangering the trustworthiness of the models and follow-up decision making
based on their confidence. We propose four typical forms of calibration
attacks: underconfidence, overconfidence, maximum miscalibration, and random
confidence attacks, conducted in both the black-box and white-box setups. We
demonstrate that the attacks are highly effective on both convolutional and
attention-based models: with a small number of queries, they seriously skew
confidence without changing the predictive performance. Given the potential
danger, we further investigate the effectiveness of a wide range of adversarial
defence and recalibration methods, including our proposed defences specifically
designed for calibration attacks to mitigate the harm. From the ECE and KS
scores, we observe that there are still significant limitations in handling
calibration attacks. To the best of our knowledge, this is the first dedicated
study that provides a comprehensive investigation on calibration-focused
attacks. We hope this study helps attract more attention to these types of
attacks and hence hamper their potential serious damages. To this end, this
work also provides detailed analyses to understand the characteristics of the
attacks.
