  Rain removal is important for improving the robustness of outdoor vision
based systems. Current rain removal methods show limitations either for complex
dynamic scenes shot from fast moving cameras, or under torrential rain fall
with opaque occlusions. We propose a novel derain algorithm, which applies
superpixel (SP) segmentation to decompose the scene into depth consistent
units. Alignment of scene contents are done at the SP level, which proves to be
robust towards rain occlusion and fast camera motion. Two alignment output
tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match
tensor, provide informative clues for rain streak location and occluded
background contents to generate an intermediate derain output. These tensors
will be subsequently prepared as input features for a convolutional neural
network to restore high frequency details to the intermediate output for
compensation of mis-alignment blur. Extensive evaluations show that up to 5 dB
reconstruction PSNR advantage is achieved over state-of-the-art methods. Visual
inspection shows that much cleaner rain removal is achieved especially for
highly dynamic scenes with heavy and opaque rainfall from a fast moving camera.
