  Neural machine translation is known to require large numbers of parallel
training sentences, which generally prevent it from excelling on low-resource
language pairs. This thesis explores the use of cross-lingual transfer learning
on neural networks as a way of solving the problem with the lack of resources.
We propose several transfer learning approaches to reuse a model pretrained on
a high-resource language pair. We pay particular attention to the simplicity of
the techniques. We study two scenarios: (a) when we reuse the high-resource
model without any prior modifications to its training process and (b) when we
can prepare the first-stage high-resource model for transfer learning in
advance. For the former scenario, we present a proof-of-concept method by
reusing a model trained by other researchers. In the latter scenario, we
present a method which reaches even larger improvements in translation
performance. Apart from proposed techniques, we focus on an in-depth analysis
of transfer learning techniques and try to shed some light on transfer learning
improvements. We show how our techniques address specific problems of
low-resource languages and are suitable even in high-resource transfer
learning. We evaluate the potential drawbacks and behavior by studying transfer
learning in various situations, for example, under artificially damaged
training corpora, or with fixed various model parts.
