  Domain generalization (DG) aims to train a model from limited source domains,
allowing it to generalize to unknown target domains. Typically, DG models only
employ large-scale pre-trained models during the initialization of fine-tuning.
However, large-scale pre-trained models already possess the ability to resist
domain shift. If we reference pre-trained models continuously during
fine-tuning to maintain this ability, it could further enhance the
generalization ability of the DG model. For this purpose, we introduce a new
method called Fine-Tune with Large-scale pre-trained Priors (FT-LP), which
incorporates the pre-trained model as a prior into the DG fine-tuning process,
ensuring that the model refers to its pre-trained model at each optimization
step. FT-LP comprises a theoretical framework and a simple implementation
strategy. In theory, we verify the rationality of FT-LP by introducing a
generalization error bound with the pre-trained priors for DG. In
implementation, we utilize an encoder to simulate the model distribution,
enabling the use of FT-LP when only pre-trained weights are available. In
summary, we offer a new fine-tuning method for DG algorithms to utilize
pre-trained models throughout the fine-tuning process. Through experiments on
various datasets and DG models, our proposed method exhibits significant
improvements, indicating its effectiveness.
