  Named entity recognition (NER) from text has been a widely studied problem
and usually extracts semantic information from text. Until now, NER from speech
is mostly studied in a two-step pipeline process that includes first applying
an automatic speech recognition (ASR) system on an audio sample and then
passing the predicted transcript to a NER tagger. In such cases, the error does
not propagate from one step to another as both the tasks are not optimized in
an end-to-end (E2E) fashion. Recent studies confirm that integrated approaches
(e.g., E2E ASR) outperform sequential ones (e.g., phoneme based ASR). In this
paper, we introduce a first publicly available NER annotated dataset for
English speech and present an E2E approach, which jointly optimizes the ASR and
NER tagger components. Experimental results show that the proposed E2E approach
outperforms the classical two-step approach. We also discuss how NER from
speech can be used to handle out of vocabulary (OOV) words in an ASR system.
