  Digital Object Identifiers (DOIs) are regarded as persistent; however, they
are sometimes deleted. Deleted DOIs are an important issue not only for
persistent access to scholarly content but also for bibliometrics, because they
may cause problems in correctly identifying scholarly articles. However, little
is known about how much of deleted DOIs and what causes them. We identified
deleted DOIs by comparing the datasets of all Crossref DOIs on two different
dates, investigated the number of deleted DOIs in the scholarly content along
with the corresponding document types, and analyzed the factors that cause
deleted DOIs. Using the proposed method, 708,282 deleted DOIs were identified.
The majority corresponded to individual scholarly articles such as journal
articles, proceedings articles, and book chapters. There were cases of many
DOIs assigned to the same content, e.g., retracted journal articles and
abstracts of international conferences. We show the publishers and academic
societies which are the most common in deleted DOIs. In addition, the top cases
of single scholarly content with a large number of deleted DOIs were revealed.
The findings of this study are useful for citation analysis and altmetrics, as
well as for avoiding deleted DOIs.
