  Sequential LSTM has been extended to model tree structures, giving
competitive results for a number of tasks. Existing methods model constituent
trees by bottom-up combinations of constituent nodes, making direct use of
input word information only for leaf nodes. This is different from sequential
LSTMs, which contain reference to input words for each node. In this paper, we
propose a method for automatic head-lexicalization for tree-structure LSTMs,
propagating head words from leaf nodes to every constituent node. In addition,
enabled by head lexicalization, we build a tree LSTM in the top-down direction,
which corresponds to bidirectional sequential LSTM structurally. Experiments
show that both extensions give better representations of tree structures. Our
final model gives the best results on the Standford Sentiment Treebank and
highly competitive results on the TREC question type classification task.
