  We develop a model of perceptual similarity judgment based on re-training a
deep convolution neural network (DCNN) that learns to associate different views
of each 3D object to capture the notion of object persistence and continuity in
our visual experience. The re-training process effectively performs distance
metric learning under the object persistency constraints, to modify the
view-manifold of object representations. It reduces the effective distance
between the representations of different views of the same object without
compromising the distance between those of the views of different objects,
resulting in the untangling of the view-manifolds between individual objects
within the same category and across categories. This untangling enables the
model to discriminate and recognize objects within the same category,
independent of viewpoints. We found that this ability is not limited to the
trained objects, but transfers to novel objects in both trained and untrained
categories, as well as to a variety of completely novel artificial synthetic
objects. This transfer in learning suggests the modification of distance
metrics in view- manifolds is more general and abstract, likely at the levels
of parts, and independent of the specific objects or categories experienced
during training. Interestingly, the resulting transformation of feature
representation in the deep networks is found to significantly better match
human perceptual similarity judgment than AlexNet, suggesting that object
persistence could be an important constraint in the development of perceptual
similarity judgment in biological neural networks.
