  Chemical patent documents describe a broad range of applications holding key
reaction and compound information, such as chemical structure, reaction
formulas, and molecular properties. These informational entities should be
first identified in text passages to be utilized in downstream tasks. Text
mining provides means to extract relevant information from chemical patents
through information extraction techniques. As part of the Information
Extraction task of the Cheminformatics Elsevier Melbourne University challenge,
in this work we study the effectiveness of contextualized language models to
extract reaction information in chemical patents. We assess transformer
architectures trained on a generic and specialised corpora to propose a new
ensemble model. Our best model, based on a majority ensemble approach, achieves
an exact F1-score of 92.30% and a relaxed F1-score of 96.24%. The results show
that ensemble of contextualized language models can provide an effective method
to extract information from chemical patents.
