  In existing image classification systems that use deep neural networks, the
knowledge needed for image classification is implicitly stored in model
parameters. If users want to update this knowledge, then they need to fine-tune
the model parameters. Moreover, users cannot verify the validity of inference
results or evaluate the contribution of knowledge to the results. In this
paper, we investigate a system that stores knowledge for image classification,
such as image feature maps, labels, and original images, not in model
parameters but in external high-capacity storage. Our system refers to the
storage like a database when classifying input images. To increase knowledge,
our system updates the database instead of fine-tuning model parameters, which
avoids catastrophic forgetting in incremental learning scenarios. We revisit a
kNN (k-Nearest Neighbor) classifier and employ it in our system. By analyzing
the neighborhood samples referred by the kNN algorithm, we can interpret how
knowledge learned in the past is used for inference results. Our system
achieves 79.8% top-1 accuracy on the ImageNet dataset without fine-tuning model
parameters after pretraining, and 90.8% accuracy on the Split CIFAR-100 dataset
in the task incremental learning setting.
