  Large-scale Vision-Language Models (VLMs) have demonstrated exceptional
performance in natural vision tasks, motivating researchers across domains to
explore domain-specific VLMs. However, the construction of powerful
domain-specific VLMs demands vast amounts of annotated data, substantial
electrical energy, and computing resources, primarily accessible to industry,
yet hindering VLM research in academia. To address this challenge and foster
sustainable and equitable VLM research, we present the Generalized Domain
Prompt Learning (GDPL) framework. GDPL facilitates the transfer of VLMs' robust
recognition capabilities from natural vision to specialized domains, without
the need for extensive data or resources. By leveraging small-scale
domain-specific foundation models and minimal prompt samples, GDPL empowers the
language branch with domain knowledge through quaternion networks, uncovering
cross-modal relationships between domain-specific vision features and natural
vision-based contextual embeddings. Simultaneously, GDPL guides the vision
branch into specific domains through hierarchical propagation of generated
vision prompt features, grounded in well-matched vision-language relations.
Furthermore, to fully harness the domain adaptation potential of VLMs, we
introduce a novel low-rank adaptation approach. Extensive experiments across
diverse domains like remote sensing, medical imaging, geology, Synthetic
Aperture Radar, and fluid dynamics, validate the efficacy of GDPL,
demonstrating its ability to achieve state-of-the-art domain recognition
performance in a prompt learning paradigm. Our framework paves the way for
sustainable and inclusive VLM research, transcending the barriers between
academia and industry.
