  In this paper, we study the problem of efficiently assessing building damage
after natural disasters like hurricanes, floods or fires, through aerial video
analysis. We make two main contributions. The first contribution is a new
dataset, consisting of user-generated aerial videos from social media with
annotations of instance-level building damage masks. This provides the first
benchmark for quantitative evaluation of models to assess building damage using
aerial videos. The second contribution is a new model, namely MSNet, which
contains novel region proposal network designs and an unsupervised score
refinement network for confidence score calibration in both bounding box and
mask branches. We show that our model achieves state-of-the-art results
compared to previous methods in our dataset. We will release our data, models
and code.
