  It has been reliably shown that the similarity of word embeddings obtained
from popular neural models such as BERT approximates effectively a form of
semantic similarity of the meaning of those words. It is therefore natural to
wonder if those embeddings contain enough information to be able to connect
those meanings through ontological relationships such as the one of
subsumption. If so, large knowledge models could be built that are capable of
semantically relating terms based on the information encapsulated in word
embeddings produced by pre-trained models, with implications not only for
ontologies (ontology matching, ontology evolution, etc.) but also on the
ability to integrate ontological knowledge in neural models. In this paper, we
test how embeddings produced by several pre-trained models can be used to
predict relations existing between classes and properties of popular
upper-level and general ontologies. We show that even a simple feed-forward
architecture on top of those embeddings can achieve promising accuracies, with
varying generalisation abilities depending on the input data. To achieve that,
we produce a dataset that can be used to further enhance those models, opening
new possibilities for applications integrating knowledge from web ontologies.
