  Transfer learning is aimed to make use of valuable knowledge in a source
domain to help model performance in a target domain. It is particularly
important to neural networks, which are very likely to be overfitting. In some
fields like image processing, many studies have shown the effectiveness of
neural network-based transfer learning. For neural NLP, however, existing
studies have only casually applied transfer learning, and conclusions are
inconsistent. In this paper, we conduct systematic case studies and provide an
illuminating picture on the transferability of neural networks in NLP.
