  Neural systems, artificial and biological, show similar representations of
inputs when optimized to perform similar tasks. In visual systems optimized for
tasks similar to object recognition, we propose that representation
similarities arise from the constraints imposed by the development of
abstractions in the representation across the processing stages. To study the
effect of abstraction hierarchy of representations across different visual
systems, we constructed a two-dimensional space in which each neural
representation is positioned based on its distance from the pixel space and the
class space. Trajectories of representations in all the task-optimized visual
neural networks start close to the pixel space and gradually move towards
higher abstract representations, such as object categories. We also observe
that proximity in this abstraction space predicts the similarity of neural
representations between visual systems. The gradual similar change of the
representations suggests that the similarity across different task-optimized
systems could arise from constraints on representational trajectories.
