  Image restoration aims to reconstruct the latent clear images from their
degraded versions. Despite the notable achievement, existing methods
predominantly focus on handling specific degradation types and thus require
specialized models, impeding real-world applications in dynamic degradation
scenarios. To address this issue, we propose Large Model Driven Image
Restoration framework (LMDIR), a novel multiple-in-one image restoration
paradigm that leverages the generic priors from large multi-modal language
models (MMLMs) and the pretrained diffusion models. In detail, LMDIR integrates
three key prior knowledges: 1) global degradation knowledge from MMLMs, 2)
scene-aware contextual descriptions generated by MMLMs, and 3) fine-grained
high-quality reference images synthesized by diffusion models guided by MMLM
descriptions. Standing on above priors, our architecture comprises a
query-based prompt encoder, degradation-aware transformer block injecting
global degradation knowledge, content-aware transformer block incorporating
scene description, and reference-based transformer block incorporating
fine-grained image priors. This design facilitates single-stage training
paradigm to address various degradations while supporting both automatic and
user-guided restoration. Extensive experiments demonstrate that our designed
method outperforms state-of-the-art competitors on multiple evaluation
benchmarks.
