  Computational argumentation has become an essential tool in various domains,
including law, public policy, and artificial intelligence. It is an emerging
research field in natural language processing that attracts increasing
attention. Research on computational argumentation mainly involves two types of
tasks: argument mining and argument generation. As large language models (LLMs)
have demonstrated impressive capabilities in understanding context and
generating natural language, it is worthwhile to evaluate the performance of
LLMs on diverse computational argumentation tasks. This work aims to embark on
an assessment of LLMs, such as ChatGPT, Flan models, and LLaMA2 models, in both
zero-shot and few-shot settings. We organize existing tasks into six main
categories and standardize the format of fourteen openly available datasets. In
addition, we present a new benchmark dataset on counter speech generation that
aims to holistically evaluate the end-to-end performance of LLMs on argument
mining and argument generation. Extensive experiments show that LLMs exhibit
commendable performance across most of the datasets, demonstrating their
capabilities in the field of argumentation. Our analysis offers valuable
suggestions for evaluating computational argumentation and its integration with
LLMs in future research endeavors.
