  We present a causal view on the robustness of neural networks against input
manipulations, which applies not only to traditional classification tasks but
also to general measurement data. Based on this view, we design a deep causal
manipulation augmented model (deep CAMA) which explicitly models possible
manipulations on certain causes leading to changes in the observed effect. We
further develop data augmentation and test-time fine-tuning methods to improve
deep CAMA's robustness. When compared with discriminative deep neural networks,
our proposed model shows superior robustness against unseen manipulations. As a
by-product, our model achieves disentangled representation which separates the
representation of manipulations from those of other latent causes.
