  Long Short-Term Memory recurrent neural network (LSTM) is widely used and
known to capture informative long-term syntactic dependencies. However, how
such information are reflected in its internal vectors for natural text has not
yet been sufficiently investigated. We analyze them by learning a language
model where syntactic structures are implicitly given. We empirically show that
the context update vectors, i.e. outputs of internal gates, are approximately
quantized to binary or ternary values to help the language model to count the
depth of nesting accurately, as Suzgun et al. (2019) recently show for
synthetic Dyck languages. For some dimensions in the context vector, we show
that their activations are highly correlated with the depth of phrase
structures, such as VP and NP. Moreover, with an $L_1$ regularization, we also
found that it can accurately predict whether a word is inside a phrase
structure or not from a small number of components of the context vector. Even
for the case of learning from raw text, context vectors are shown to still
correlate well with the phrase structures. Finally, we show that natural
clusters of the functional words and the part of speeches that trigger phrases
are represented in a small but principal subspace of the context-update vector
of LSTM.
