  This paper studies the relationship between the classification performed by
deep neural networks (DNNs) and the decision of various classical classifiers,
namely k-nearest neighbours (k-NN), support vector machines (SVM) and logistic
regression (LR), at various layers of the network. This comparison provides us
with new insights as to the ability of neural networks to both memorize the
training data and generalize to new data at the same time, where k-NN serves as
the ideal estimator that perfectly memorizes the data. We show that
memorization of non-generalizing networks happens only at the last layers.
Moreover, the behavior of DNNs compared to the linear classifiers SVM and LR is
quite the same on the training and test data regardless of whether the network
generalizes. On the other hand, the similarity to k-NN holds only at the
absence of overfitting. Our results suggests that k-NN behavior of the network
on new data is a sign of generalization. Moreover, it shows that memorization
and generalization, which are traditionally considered to be contradicting to
each other, are compatible and complementary.
