  Missingness, or the absence of features from an input, is a concept
fundamental to many model debugging tools. However, in computer vision, pixels
cannot simply be removed from an image. One thus tends to resort to heuristics
such as blacking out pixels, which may in turn introduce bias into the
debugging process. We study such biases and, in particular, show how
transformer-based architectures can enable a more natural implementation of
missingness, which side-steps these issues and improves the reliability of
model debugging in practice. Our code is available at
https://github.com/madrylab/missingness
