  Pre-trained vision and language models such as CLIP have witnessed remarkable
success in connecting images and texts with a primary focus on English texts.
Despite recent efforts to extend CLIP to support other languages, disparities
in performance among different languages have been observed due to uneven
resource availability. Additionally, current cross-lingual transfer methods of
those pre-trained models would consume excessive resources for a large number
of languages. Therefore, we propose a new parameter-efficient cross-lingual
transfer learning framework that utilizes a translation-based alignment method
to mitigate multilingual disparities and explores parameter-efficient
fine-tuning methods for parameter-efficient cross-lingual transfer. Extensive
experiments on XTD and Multi30K datasets, covering 11 languages under
zero-shot, few-shot, and full-dataset learning scenarios, show that our
framework significantly reduces the multilingual disparities among languages
and improves cross-lingual transfer results, especially in low-resource
scenarios, while only keeping and fine-tuning an extremely small number of
parameters compared to the full model (e.g., Our framework only requires 0.16\%
additional parameters of a full-model for each language in the few-shot
learning scenario). The codes are available at
\url{https://github.com/eric-ai-lab/PECTVLM}. The codes are available at
\url{https://github.com/eric-ai-lab/PECTVLM}.
