  In this paper we present a set of experiments carried out with BERT on a
number of Italian sentences taken from poetry domain. The experiments are
organized on the hypothesis of a very high level of difficulty in
predictability at the three levels of linguistic complexity that we intend to
monitor: lexical, syntactic and semantic level. To test this hypothesis we ran
the Italian version of BERT with 80 sentences for a total of 900 tokens mostly
extracted from Italian poetry of the first half of last century. Then we
alternated canonical and noncanonical versions of the same sentence before
processing them with the same DL model. We used then sentences from the
newswire domain containing similar syntactic structures. The results show that
the DL model is highly sensitive to presence of noncanonical structures.
However, DLs are also very sensitive to word frequency and to local non literal
meaning compositional effect. This is also apparent by the preference for
predicting function vs content words, collocates vs infrequent word phrases. In
the paper, we focused our attention on the use of subword units done by BERT
for out of vocabulary words.
