  With the increasing diversity of use cases of large language models, a more
informative treatment of texts seems necessary. An argumentative analysis could
foster a more reasoned usage of chatbots, text completion mechanisms or other
applications. However, it is unclear which aspects of argumentation can be
reliably identified and integrated in language models. In this paper, we
present an empirical assessment of the reliability with which different
argumentative aspects can be automatically identified in hate speech in social
media. We have enriched the Hateval corpus (Basile et al. 2019) with a manual
annotation of some argumentative components, adapted from Wagemans (2016)'s
Periodic Table of Arguments. We show that some components can be identified
with reasonable reliability. For those that present a high error ratio, we
analyze the patterns of disagreement between expert annotators and errors in
automatic procedures, and we propose adaptations of those categories that can
be more reliably reproduced.
