  The inspection and monitoring of infrastructure assets typically requires
identifying visual anomalies in scenes periodically photographed over time.
Images collected manually or with robots such as unmanned aerial vehicles from
the same scene at different instances in time are typically not perfectly
aligned. Supervised segmentation methods can be applied to identify known
problems, but unsupervised anomaly detection approaches are required when
unknown anomalies occur. Current unsupervised pixel-level anomaly detection
methods have mainly been developed for industrial settings where the camera
position is known and constant. However, we find that these methods fail to
generalize to the case when images are not perfectly aligned. We term the
problem of unsupervised anomaly detection between two such imperfectly aligned
sets of images as Scene Anomaly Detection (Scene AD). We present a novel
network termed OmniAD to address the Scene AD problem posed. Specifically, we
refine the anomaly detection method reverse distillation to achieve a 40%
increase in pixel-level anomaly detection performance. The network's
performance is further demonstrated to improve with two new data augmentation
strategies proposed that leverage novel view synthesis and camera localization
to improve generalization. We validate our approach with qualitative and
quantitative results on a new dataset, ToyCity, the first Scene AD dataset with
multiple objects, as well as on the established single object-centric dataset,
MAD. https://drags99.github.io/OmniAD/
