  Recent progress in cross-lingual relation and event extraction use graph
convolutional networks (GCNs) with universal dependency parses to learn
language-agnostic sentence representations such that models trained on one
language can be applied to other languages. However, GCNs struggle to model
words with long-range dependencies or are not directly connected in the
dependency tree. To address these challenges, we propose to utilize the
self-attention mechanism where we explicitly fuse structural information to
learn the dependencies between words with different syntactic distances. We
introduce GATE, a {\bf G}raph {\bf A}ttention {\bf T}ransformer {\bf E}ncoder,
and test its cross-lingual transferability on relation and event extraction
tasks. We perform experiments on the ACE05 dataset that includes three
typologically different languages: English, Chinese, and Arabic. The evaluation
results show that GATE outperforms three recently proposed methods by a large
margin. Our detailed analysis reveals that due to the reliance on syntactic
dependencies, GATE produces robust representations that facilitate transfer
across languages.
