  Multi-task learning (MTL) has achieved remarkable success in natural language
processing applications. In this work, we study a multi-task learning model
with multiple decoders on varieties of biomedical and clinical natural language
processing tasks such as text similarity, relation extraction, named entity
recognition, and text inference. Our empirical results demonstrate that the MTL
fine-tuned models outperform state-of-the-art transformer models (e.g., BERT
and its variants) by 2.0% and 1.3% in biomedical and clinical domains,
respectively. Pairwise MTL further demonstrates more details about which tasks
can improve or decrease others. This is particularly helpful in the context
that researchers are in the hassle of choosing a suitable model for new
problems. The code and models are publicly available at
https://github.com/ncbi-nlp/bluebert
