  Extracting relational triples from text is a crucial task for constructing
knowledge bases. Recent advancements in joint entity and relation extraction
models have demonstrated remarkable F1 scores ($\ge 90\%$) in accurately
extracting relational triples from free text. However, these models have been
evaluated under restrictive experimental settings and unrealistic datasets.
They overlook sentences with zero triples (zero-cardinality), thereby
simplifying the task. In this paper, we present a benchmark study of
state-of-the-art joint entity and relation extraction models under a more
realistic setting. We include sentences that lack any triples in our
experiments, providing a comprehensive evaluation. Our findings reveal a
significant decline (approximately 10-15\% in one dataset and 6-14\% in another
dataset) in the models' F1 scores within this realistic experimental setup.
Furthermore, we propose a two-step modeling approach that utilizes a simple
BERT-based classifier. This approach leads to overall performance improvement
in these models within the realistic experimental setting.
