  Deep neural networks (DNNs) have shown the state-of-the-art level of
performances in wide range of complicated tasks. In recent years, the studies
have been actively conducted to analyze the black box characteristics of DNNs
and to grasp the learning behaviours, tendency, and limitations of DNNs. In
this paper, we investigate the limitation of DNNs in image classification task
and verify it with the method inspired by cognitive psychology. Through
analyzing the failure cases of ImageNet classification task, we hypothesize
that the DNNs do not sufficiently learn to associate related classes of
objects. To verify how DNNs understand the relatedness between object classes,
we conducted experiments on the image database provided in cognitive
psychology. We applied the ImageNet-trained DNNs to the database consisting of
pairs of related and unrelated object images to compare the feature
similarities and determine whether the pairs match each other. In the
experiments, we observed that the DNNs show limited performance in determining
relatedness between object classes. In addition, the DNNs present somewhat
improved performance in discovering relatedness based on similarity, but they
perform weaker in discovering relatedness based on association. Through these
experiments, a novel analysis of learning behaviour of DNNs is provided and the
limitation which needs to be overcome is suggested.
