  Rain streaks showing in images or videos would severely degrade the
performance of computer vision applications. Thus, it is of vital importance to
remove rain streaks and facilitate our vision systems. While recent
convolutinal neural network based methods have shown promising results in
single image rain removal (SIRR), they fail to effectively capture long-range
location dependencies or aggregate convolutional channel information
simultaneously. However, as SIRR is a highly illposed problem, these spatial
and channel information are very important clues to solve SIRR. First, spatial
information could help our model to understand the image context by gathering
long-range dependency location information hidden in the image. Second,
aggregating channels could help our model to concentrate on channels more
related to image background instead of rain streaks. In this paper, we propose
a non-local channel aggregation network (NCANet) to address the SIRR problem.
NCANet models 2D rainy images as sequences of vectors in three directions,
namely vertical direction, transverse direction and channel direction.
Recurrently aggregating information from all three directions enables our model
to capture the long-range dependencies in both channels and spaitials
locations. Extensive experiments on both heavy and light rain image data sets
demonstrate the effectiveness of the proposed NCANet model.
