  Temporal information extraction (TIE) has attracted a great deal of interest
over the last two decades, leading to the development of a significant number
of datasets. Despite its benefits, having access to a large volume of corpora
makes it difficult when it comes to benchmark TIE systems. On the one hand,
different datasets have different annotation schemes, thus hindering the
comparison between competitors across different corpora. On the other hand, the
fact that each corpus is commonly disseminated in a different format requires a
considerable engineering effort for a researcher/practitioner to develop
parsers for all of them. This constraint forces researchers to select a limited
amount of datasets to evaluate their systems which consequently limits the
comparability of the systems. Yet another obstacle that hinders the
comparability of the TIE systems is the evaluation metric employed. While most
research works adopt traditional metrics such as precision, recall, and $F_1$,
a few others prefer temporal awareness -- a metric tailored to be more
comprehensive on the evaluation of temporal systems. Although the reason for
the absence of temporal awareness in the evaluation of most systems is not
clear, one of the factors that certainly weights this decision is the necessity
to implement the temporal closure algorithm in order to compute temporal
awareness, which is not straightforward to implement neither is currently
easily available. All in all, these problems have limited the fair comparison
between approaches and consequently, the development of temporal extraction
systems. To mitigate these problems, we have developed tieval, a Python library
that provides a concise interface for importing different corpora and
facilitates system evaluation. In this paper, we present the first public
release of tieval and highlight its most relevant features.
