  Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting
novel categories from a distinct domain using only limited exemplars. In this
paper, we undertake a comprehensive study of CD-FSS and uncover two crucial
insights: (i) the necessity of a fine-tuning stage to effectively transfer the
learned meta-knowledge across domains, and (ii) the overfitting risk during the
na\"ive fine-tuning due to the scarcity of novel category examples. With these
insights, we propose a novel cross-domain fine-tuning strategy that addresses
the challenging CD-FSS tasks. We first design Bi-directional Few-shot
Prediction (BFP), which establishes support-query correspondence in a
bi-directional manner, crafting augmented supervision to reduce the overfitting
risk. Then we further extend BFP into Iterative Few-shot Adaptor (IFA), which
is a recursive framework to capture the support-query correspondence
iteratively, targeting maximal exploitation of supervisory signals from the
sparse novel category samples. Extensive empirical evaluations show that our
method significantly outperforms the state-of-the-arts (+7.8\%), which verifies
that IFA tackles the cross-domain challenges and mitigates the overfitting
simultaneously. The code is available at: https://github.com/niejiahao1998/IFA.
