  This paper is a preliminary study of the robustness and noise analysis of
deep neural networks via a game theory formulation Bayesian Neural Networks
(BNN) and the maximal coding rate distortion loss. BNN has been shown to
provide some robustness to deep learning, and the minimax method used to be a
natural conservative way to assist the Bayesian method. Inspired by the recent
closed-loop transcription neural network, we formulate the BNN via game theory
between the deterministic neural network $f$ and the sampling network $f + \xi$
or $f + r*\xi$. Compared with previous BNN, BNN via game theory learns a
solution space within a certain gap between the center $f$ and the sampling
point $f + r*\xi$, and is a conservative choice with a meaningful prior setting
compared with previous BNN. Furthermore, the minimum points between $f$ and $f
+ r*\xi$ become stable when the subspace dimension is large enough with a
well-trained model $f$. With these, the model $f$ can have a high chance of
recognizing the out-of-distribution data or noise data in the subspace rather
than the prediction level, even if $f$ is in online training after a few
iterations of true data. So far, our experiments are limited to MNIST and
Fashion MNIST data sets, more experiments with realistic data sets and
complicated neural network models should be implemented to validate the above
arguments.
