  Natural disasters ravage the world's cities, valleys, and shores on a regular
basis. Deploying precise and efficient computational mechanisms for assessing
infrastructure damage is essential to channel resources and minimize the loss
of life. Using a dataset that includes labeled pre- and post- disaster
satellite imagery, we take a machine learning-based remote sensing approach and
train multiple convolutional neural networks (CNNs) to assess building damage
on a per-building basis. We present a novel methodology of interpretable deep
learning that seeks to explicitly investigate the most useful modalities of
information in the training data to create an accurate classification model. We
also investigate which loss functions best optimize these models. Our findings
include that ordinal-cross entropy loss is the most optimal criterion for
optimization to use and that including the type of disaster that caused the
damage in combination with pre- and post-disaster training data most accurately
predicts the level of damage caused. Further, we make progress in the
qualitative representation of which parts of the images that the model is using
to predict damage levels, through gradient-weighted class activation mapping
(Grad-CAM). Our research seeks to computationally contribute to aiding in this
ongoing and growing humanitarian crisis, heightened by anthropogenic climate
change.
