  In recent years, deep neural language models have made strong progress in
various NLP tasks. This work explores one facet of the question whether
state-of-the-art NLP models exhibit elementary mechanisms known from human
cognition. The exploration is focused on a relatively primitive mechanism for
which there is a lot of evidence from various psycholinguistic experiments with
infants. The computation of "abstract sameness relations" is assumed to play an
important role in human language acquisition and processing, especially in
learning more complex grammar rules. In order to investigate this mechanism in
BERT and other pre-trained language models (PLMs), the experiment designs from
studies with infants were taken as the starting point. On this basis, we
designed experimental settings in which each element from the original studies
was mapped to a component of language models. Even though the task in our
experiments was relatively simple, the results suggest that the cognitive
faculty of computing abstract sameness relations is stronger in infants than in
all investigated PLMs.
