  Automatic translation systems are known to struggle with rare words. Among
these, named entities (NEs) and domain-specific terms are crucial, since errors
in their translation can lead to severe meaning distortions. Despite their
importance, previous speech translation (ST) studies have neglected them, also
due to the dearth of publicly available resources tailored to their specific
evaluation. To fill this gap, we i) present the first systematic analysis of
the behavior of state-of-the-art ST systems in translating NEs and terminology,
and ii) release NEuRoparl-ST, a novel benchmark built from European Parliament
speeches annotated with NEs and terminology. Our experiments on the three
language directions covered by our benchmark (en->es/fr/it) show that ST
systems correctly translate 75-80% of terms and 65-70% of NEs, with very low
performance (37-40%) on person names.
