  We present a novel approach to detecting noun abstraction within a large
language model (LLM). Starting from a psychologically motivated set of noun
pairs in taxonomic relationships, we instantiate surface patterns indicating
hypernymy and analyze the attention matrices produced by BERT. We compare the
results to two sets of counterfactuals and show that we can detect hypernymy in
the abstraction mechanism, which cannot solely be related to the distributional
similarity of noun pairs. Our findings are a first step towards the
explainability of conceptual abstraction in LLMs.
