  In this article we report on an initial exploration to assess the viability
of using the general large language models (LLMs), recently made public, to
classify mathematical documents. Automated classification would be useful from
the applied perspective of improving the navigation of the literature and the
more open-ended goal of identifying relations among mathematical results. The
Mathematical Subject Classification MSC 2020, from MathSciNet and zbMATH, is
widely used and there is a significant corpus of ground truth material in the
open literature. We have evaluated the classification of preprint articles from
arXiv.org according to MSC 2020. The experiment used only the title and
abstract alone -- not the entire paper. Since this was early in the use of
chatbots and the development of their APIs, we report here on what was carried
out by hand. Of course, the automation of the process will have to follow if it
is to be generally useful. We found that in about 60% of our sample the LLM
produced a primary classification matching that already reported on arXiv. In
about half of those instances, there were additional primary classifications
that were not detected. In about 40% of our sample, the LLM suggested a
different classification than what was provided. A detailed examination of
these cases, however, showed that the LLM-suggested classifications were in
most cases better than those provided.
