  Extraction of relevant pathological terms from radiology reports is important
for correct image label generation and disease population studies. In this
letter, we compare the performance of some known application program interface
(APIs) for the task of thoracic abnormality extraction from radiology reports.
We explored several medical domain specific annotation tools like Medical Text
Indexer(MTI) with Non-MEDLINE and Mesh On Demand(MOD) options and generic
Natural Language Understanding (NLU) API provided by the IBM cloud. Our results
show that although MTI and MOD are intended for extracting medical terms, their
performance is worst compared to generic extraction API like IBM NLU. Finally,
we trained a DNN-based Named Entity Recognition (NER) model to extract the key
concept words from radiology reports. Our model outperforms the medical
specific and generic API performance by a large margin. Our results demonstrate
the inadequacy of generic APIs for pathology extraction task and establish the
importance of domain specific model training for improved results. We hope that
these results motivate the research community to release larger de-identified
radiology reports corpus for building high accuracy machine learning models for
the important task of pathology extraction.
