  Blind-spot network (BSN) and its variants have made significant advances in
self-supervised denoising. Nevertheless, they are still bound to synthetic
noisy inputs due to less practical assumptions like pixel-wise independent
noise. Hence, it is challenging to deal with spatially correlated real-world
noise using self-supervised BSN. Recently, pixel-shuffle downsampling (PD) has
been proposed to remove the spatial correlation of real-world noise. However,
it is not trivial to integrate PD and BSN directly, which prevents the fully
self-supervised denoising model on real-world images. We propose an Asymmetric
PD (AP) to address this issue, which introduces different PD stride factors for
training and inference. We systematically demonstrate that the proposed AP can
resolve inherent trade-offs caused by specific PD stride factors and make BSN
applicable to practical scenarios. To this end, we develop AP-BSN, a
state-of-the-art self-supervised denoising method for real-world sRGB images.
We further propose random-replacing refinement, which significantly improves
the performance of our AP-BSN without any additional parameters. Extensive
studies demonstrate that our method outperforms the other self-supervised and
even unpaired denoising methods by a large margin, without using any additional
knowledge, e.g., noise level, regarding the underlying unknown noise.
