  Recurrent Neural Networks (RNNs) have achieved tremendous success in
sequential data processing. However, it is quite challenging to interpret and
verify RNNs' behaviors directly. To this end, many efforts have been made to
extract finite automata from RNNs. Existing approaches such as exact learning
are effective in extracting finite-state models to characterize the state
dynamics of RNNs for formal languages, but are limited in the scalability to
process natural languages. Compositional approaches that are scablable to
natural languages fall short in extraction precision. In this paper, we
identify the transition sparsity problem that heavily impacts the extraction
precision. To address this problem, we propose a transition rule extraction
approach, which is scalable to natural language processing models and effective
in improving extraction precision. Specifically, we propose an empirical method
to complement the missing rules in the transition diagram. In addition, we
further adjust the transition matrices to enhance the context-aware ability of
the extracted weighted finite automaton (WFA). Finally, we propose two data
augmentation tactics to track more dynamic behaviors of the target RNN.
Experiments on two popular natural language datasets show that our method can
extract WFA from RNN for natural language processing with better precision than
existing approaches. Our code is available at
https://github.com/weizeming/Extract_WFA_from_RNN_for_NL.
