  This article explores the adaptive relationship between Encoder Layers and
Decoder Layers using the SOTA model Helsinki-NLP/opus-mt-de-en, which
translates German to English. The specific method involves introducing a
bias-free fully connected layer between the Encoder and Decoder, with different
initializations of the layer's weights, and observing the outcomes of
fine-tuning versus retraining. Four experiments were conducted in total. The
results suggest that directly modifying the pre-trained model structure for
fine-tuning yields suboptimal performance. However, upon observing the outcomes
of the experiments with retraining, this structural adjustment shows
significant potential.
