  The Sentence-State LSTM (S-LSTM) is a powerful and high efficient graph
recurrent network, which views words as nodes and performs layer-wise recurrent
steps between them simultaneously. Despite its successes on text
representations, the S-LSTM still suffers from two drawbacks. Firstly, given a
sentence, certain words are usually more ambiguous than others, and thus more
computation steps need to be taken for these difficult words and vice versa.
However, the S-LSTM takes fixed computation steps for all words, irrespective
of their hardness. The secondary one comes from the lack of sequential
information (e.g., word order) that is inherently important for natural
language. In this paper, we try to address these issues and propose a
depth-adaptive mechanism for the S-LSTM, which allows the model to learn how
many computational steps to conduct for different words as required. In
addition, we integrate an extra RNN layer to inject sequential information,
which also serves as an input feature for the decision of adaptive depths.
Results on the classic text classification task (24 datasets in various sizes
and domains) show that our model brings significant improvements against the
conventional S-LSTM and other high-performance models (e.g., the Transformer),
meanwhile achieving a good accuracy-speed trade off.
