  The goal of this paper is to learn more about how idiomatic information is
structurally encoded in embeddings, using a structural probing method. We
repurpose an existing English verbal multi-word expression (MWE) dataset to
suit the probing framework and perform a comparative probing study of static
(GloVe) and contextual (BERT) embeddings. Our experiments indicate that both
encode some idiomatic information to varying degrees, but yield conflicting
evidence as to whether idiomaticity is encoded in the vector norm, leaving this
an open question. We also identify some limitations of the used dataset and
highlight important directions for future work in improving its suitability for
a probing analysis.
