  The ability to remove features from the input of machine learning models is
very important to understand and interpret model predictions. However, this is
non-trivial for vision models since masking out parts of the input image
typically causes large distribution shifts. This is because the baseline color
used for masking (typically grey or black) is out of distribution. Furthermore,
the shape of the mask itself can contain unwanted signals which can be used by
the model for its predictions. Recently, there has been some progress in
mitigating this issue (called missingness bias) in image masking for vision
transformers. In this work, we propose a new masking method for CNNs we call
layer masking in which the missingness bias caused by masking is reduced to a
large extent. Intuitively, layer masking applies a mask to intermediate
activation maps so that the model only processes the unmasked input. We show
that our method (i) is able to eliminate or minimize the influence of the mask
shape or color on the output of the model, and (ii) is much better than
replacing the masked region by black or grey for input perturbation based
interpretability techniques like LIME. Thus, layer masking is much less
affected by missingness bias than other masking strategies. We also demonstrate
how the shape of the mask may leak information about the class, thus affecting
estimates of model reliance on class-relevant features derived from input
masking. Furthermore, we discuss the role of data augmentation techniques for
tackling this problem, and argue that they are not sufficient for preventing
model reliance on mask shape. The code for this project is publicly available
at https://github.com/SriramB-98/layer_masking
