  In this paper, we propose an explanation of representation for self-attention
network (SAN) based neural sequence encoders, which regards the information
captured by the model and the encoding of the model as graph structure and the
generation of these graph structures respectively. The proposed explanation
applies to existing works on SAN-based models and can explain the relationship
among the ability to capture the structural or linguistic information, depth of
model, and length of sentence, and can also be extended to other models such as
recurrent neural network based models. We also propose a revisited multigraph
called Multi-order-Graph (MoG) based on our explanation to model the graph
structures in the SAN-based model as subgraphs in MoG and convert the encoding
of SAN-based model to the generation of MoG. Based on our explanation, we
further introduce a Graph-Transformer by enhancing the ability to capture
multiple subgraphs of different orders and focusing on subgraphs of high
orders. Experimental results on multiple neural machine translation tasks show
that the Graph-Transformer can yield effective performance improvement.
