  Recently deep neural networks based on tanh activation function have shown
their impressive power in image denoising. In this letter, we try to use
rectifier function instead of tanh and propose a dual-pathway rectifier neural
network by combining two rectifier neurons with reversed input and output
weights in the same hidden layer. We drive the equivalent activation function
and compare it to some typical activation functions for image denoising under
the same network architecture. The experimental results show that our model
achieves superior performances faster especially when the noise is small.
