  Syntax-incorporated machine translation models have been proven successful in
improving the model's reasoning and meaning preservation ability. In this
paper, we propose a simple yet effective graph-structured encoder, the
Recurrent Graph Syntax Encoder, dubbed \textbf{RGSE}, which enhances the
ability to capture useful syntactic information. The RGSE is done over a
standard encoder (recurrent or self-attention encoder), regarding recurrent
network units as graph nodes and injects syntactic dependencies as edges, such
that RGSE models syntactic dependencies and sequential information
(\textit{i.e.}, word order) simultaneously. Our approach achieves considerable
improvements over several syntax-aware NMT models in English$\Rightarrow$German
and English$\Rightarrow$Czech translation tasks. And RGSE-equipped big model
obtains competitive result compared with the state-of-the-art model in WMT14
En-De task. Extensive analysis further verifies that RGSE could benefit long
sentence modeling, and produces better translations.
