  Deep learning (DL) has proven to be an effective machine learning and
computer vision technique. DL-based image segmentation, object recognition and
classification will aid many in-situ Mars rover tasks such as path planning and
artifact recognition/extraction. However, most of the Deep Neural Network (DNN)
architectures are so complex that they are considered a 'black box'. In this
paper, we used integrated gradients to describe the attributions of each neuron
to the output classes. It provides a set of explainability tools (ET) that
opens the black box of a DNN so that the individual contribution of neurons to
category classification can be ranked and visualized. The neurons in each dense
layer are mapped and ranked by measuring expected contribution of a neuron to a
class vote given a true image label. The importance of neurons is prioritized
according to their correct or incorrect contribution to the output classes and
suppression or bolstering of incorrect classes, weighted by the size of each
class. ET provides an interface to prune the network to enhance high-rank
neurons and remove low-performing neurons. ET technology will make DNNs smaller
and more efficient for implementation in small embedded systems. It also leads
to more explainable and testable DNNs that can make systems easier for
Validation \& Verification. The goal of ET technology is to enable the adoption
of DL in future in-situ planetary exploration missions.
