  Improving the interpretability of geospatial artificial intelligence (GeoAI)
models has become critically important to open the "black box" of complex AI
models, such as deep learning. This paper compares popular saliency map
generation techniques and their strengths and weaknesses in interpreting GeoAI
and deep learning models' reasoning behaviors, particularly when applied to
geospatial analysis and image processing tasks. We surveyed two broad classes
of model explanation methods: perturbation-based and gradient-based methods.
The former identifies important image areas, which help machines make
predictions by modifying a localized area of the input image. The latter
evaluates the contribution of every single pixel of the input image to the
model's prediction results through gradient backpropagation. In this study,
three algorithms-the occlusion method, the integrated gradients method, and the
class activation map method-are examined for a natural feature detection task
using deep learning. The algorithms' strengths and weaknesses are discussed,
and the consistency between model-learned and human-understandable concepts for
object recognition is also compared. The experiments used two GeoAI-ready
datasets to demonstrate the generalizability of the research findings.
