  While lexico-semantic elements no doubt capture a large amount of linguistic
information, it has been argued that they do not capture all information
contained in text. This assumption is central to constructionist approaches to
language which argue that language consists of constructions, learned pairings
of a form and a function or meaning that are either frequent or have a meaning
that cannot be predicted from its component parts. BERT's training objectives
give it access to a tremendous amount of lexico-semantic information, and while
BERTology has shown that BERT captures certain important linguistic dimensions,
there have been no studies exploring the extent to which BERT might have access
to constructional information. In this work we design several probes and
conduct extensive experiments to answer this question. Our results allow us to
conclude that BERT does indeed have access to a significant amount of
information, much of which linguists typically call constructional information.
The impact of this observation is potentially far-reaching as it provides
insights into what deep learning methods learn from text, while also showing
that information contained in constructions is redundantly encoded in
lexico-semantics.
