  In this paper, we propose Multi$^2$OIE, which performs open information
extraction (open IE) by combining BERT with multi-head attention. Our model is
a sequence-labeling system with an efficient and effective argument extraction
method. We use a query, key, and value setting inspired by the Multimodal
Transformer to replace the previously used bidirectional long short-term memory
architecture with multi-head attention. Multi$^2$OIE outperforms existing
sequence-labeling systems with high computational efficiency on two benchmark
evaluation datasets, Re-OIE2016 and CaRB. Additionally, we apply the proposed
method to multilingual open IE using multilingual BERT. Experimental results on
new benchmark datasets introduced for two languages (Spanish and Portuguese)
demonstrate that our model outperforms other multilingual systems without
training data for the target languages.
