  One way to analyse the behaviour of machine learning models is through local
explanations that highlight input features that maximally influence model
predictions. Sensitivity analysis, which involves analysing the effect of input
perturbations on model predictions, is one of the methods to generate local
explanations. Meaningful input perturbations are essential for generating
reliable explanations, but there exists limited work on what such perturbations
are and how to perform them. This work investigates these questions in the
context of machine listening models that analyse audio. Specifically, we use a
state-of-the-art deep singing voice detection (SVD) model to analyse whether
explanations from SoundLIME (a local explanation method) are sensitive to how
the method perturbs model inputs. The results demonstrate that SoundLIME
explanations are sensitive to the content in the occluded input regions. We
further propose and demonstrate a novel method for quantitatively identifying
suitable content type(s) for reliably occluding inputs of machine listening
models. The results for the SVD model suggest that the average magnitude of
input mel-spectrogram bins is the most suitable content type for temporal
explanations.
