  The capability of Deep Neural Networks (DNNs) to recognize objects in
orientations outside the distribution of the training data is not well
understood. We present evidence that DNNs are capable of generalizing to
objects in novel orientations by disseminating orientation-invariance obtained
from familiar objects seen from many viewpoints. This capability strengthens
when training the DNN with an increasing number of familiar objects, but only
in orientations that involve 2D rotations of familiar orientations. We show
that this dissemination is achieved via neurons tuned to common features
between familiar and unfamiliar objects. These results implicate brain-like
neural mechanisms for generalization.
