  We show that unlike machine learning classifiers, there are no complex
boundary structures in the decision boundaries for well-trained deep models.
However, we found that the complicated structures do appear in training but
they vanish shortly after shaping. This is a pessimistic news if one seeks to
capture different levels of complexity in the decision boundary for
understanding generalization, which works well in machine learning.
Nonetheless, we found that the decision boundaries of predecessor models on the
training data are reflective of the final model's generalization. We show how
to use the predecessor decision boundaries for studying the generalization of
deep models. We have three major findings. One is on the strength of the first
principle component of deep models, another about the singularity of
optimizers, and the other on the effects of the skip connections in ResNets.
Code is at https://github.com/hengshu1/decision_boundary_github.
