  Large language models (LLMs) have been leveraged for several years now,
obtaining state-of-the-art performance in recognizing entities from modern
documents. For the last few months, the conversational agent ChatGPT has
"prompted" a lot of interest in the scientific community and public due to its
capacity of generating plausible-sounding answers. In this paper, we explore
this ability by probing it in the named entity recognition and classification
(NERC) task in primary sources (e.g., historical newspapers and classical
commentaries) in a zero-shot manner and by comparing it with state-of-the-art
LM-based systems. Our findings indicate several shortcomings in identifying
entities in historical text that range from the consistency of entity
annotation guidelines, entity complexity, and code-switching, to the
specificity of prompting. Moreover, as expected, the inaccessibility of
historical archives to the public (and thus on the Internet) also impacts its
performance.
