  Recent efforts have explored leveraging visible light images to enrich
texture details in infrared (IR) super-resolution. However, this direct
adaptation approach often becomes a double-edged sword, as it improves texture
at the cost of introducing noise and blurring artifacts. To address these
challenges, we propose the Target-oriented Domain Adaptation SRGAN (DASRGAN),
an innovative framework specifically engineered for robust IR super-resolution
model adaptation. DASRGAN operates on the synergy of two key components: 1)
Texture-Oriented Adaptation (TOA) to refine texture details meticulously, and
2) Noise-Oriented Adaptation (NOA), dedicated to minimizing noise transfer.
Specifically, TOA uniquely integrates a specialized discriminator,
incorporating a prior extraction branch, and employs a Sobel-guided adversarial
loss to align texture distributions effectively. Concurrently, NOA utilizes a
noise adversarial loss to distinctly separate the generative and Gaussian noise
pattern distributions during adversarial training. Our extensive experiments
confirm DASRGAN's superiority. Comparative analyses against leading methods
across multiple benchmarks and upsampling factors reveal that DASRGAN sets new
state-of-the-art performance standards. Code are available at
\url{https://github.com/yongsongH/DASRGAN}.
