  Recently, convolutional networks have achieved remarkable development in
remote sensing image Super-Resoltuion (SR) by minimizing the regression
objectives, e.g., MSE loss. However, despite achieving impressive performance,
these methods often suffer from poor visual quality with over-smooth issues.
Generative adversarial networks have the potential to infer intricate details,
but they are easy to collapse, resulting in undesirable artifacts. To mitigate
these issues, in this paper, we first introduce Diffusion Probabilistic Model
(DPM) for efficient remote sensing image SR, dubbed EDiffSR. EDiffSR is easy to
train and maintains the merits of DPM in generating perceptual-pleasant images.
Specifically, different from previous works using heavy UNet for noise
prediction, we develop an Efficient Activation Network (EANet) to achieve
favorable noise prediction performance by simplified channel attention and
simple gate operation, which dramatically reduces the computational budget.
Moreover, to introduce more valuable prior knowledge into the proposed EDiffSR,
a practical Conditional Prior Enhancement Module (CPEM) is developed to help
extract an enriched condition. Unlike most DPM-based SR models that directly
generate conditions by amplifying LR images, the proposed CPEM helps to retain
more informative cues for accurate SR. Extensive experiments on four remote
sensing datasets demonstrate that EDiffSR can restore visual-pleasant images on
simulated and real-world remote sensing images, both quantitatively and
qualitatively. The code of EDiffSR will be available at
https://github.com/XY-boy/EDiffSR
