  In scholarly publishing, blacklists aim to register fraudulent or deceptive
journals and publishers, also known as "predatory", to minimise the spread of
unreliable research and the growing of fake publishing outlets. However,
blacklisting remains a very controversial activity for several reasons: there
is no consensus regarding the criteria used to determine fraudulent journals,
the criteria used may not always be transparent or relevant, and blacklists are
rarely updated regularly. Cabell's paywalled blacklist service attempts to
overcome some of these issues in reviewing fraudulent journals on the basis of
transparent criteria and in providing allegedly up-to-date information at the
journal entry level. We tested Cabell's blacklist to analyse whether or not it
could be adopted as a reliable tool by stakeholders in scholarly communication,
including our own academic library. To do so, we used a copy of Walt Crawford's
Gray Open Access dataset (2012-2016) to assess the coverage of Cabell's
blacklist and get insights on their methodology. Out of the 10,123 journals
that we tested, 4,681 are included in Cabell's blacklist. Out of this number of
journals included in the blacklist, 3,229 are empty journals, i.e. journals in
which no single article has ever been published. Other collected data points to
questionable weighing and reviewing methods and shows a lack of rigour in how
Cabell applies its own procedures: some journals are blacklisted on the basis
of 1 to 3 criteria, identical criteria are recorded multiple times in
individual journal entries, discrepancies exist between reviewing dates and the
criteria version used and recorded by Cabell, reviewing dates are missing, and
we observed two journals blacklisted twice with a different number of
violations. Based on these observations, we conclude with recommendations and
suggestions that could help improve Cabell's blacklist service.
