  Convolutional Neural Networks (CNNs) have recently become a favored technique
for image denoising due to its adaptive learning ability, especially with a
deep configuration. However, their efficacy is inherently limited owing to
their homogenous network formation with the unique use of linear convolution.
In this study, we propose a heterogeneous network model which allows greater
flexibility for embedding additional non-linearity at the core of the data
transformation. To this end, we propose the idea of an operational neuron or
Operational Neural Networks (ONN), which enables a flexible non-linear and
heterogeneous configuration employing both inter and intra-layer neuronal
diversity. Furthermore, we propose a robust operator search strategy inspired
by the Hebbian theory, called the Synaptic Plasticity Monitoring (SPM) which
can make data-driven choices for non-linearities in any architecture. An
extensive set of comparative evaluations of ONNs and CNNs over two severe image
denoising problems yield conclusive evidence that ONNs enriched by non-linear
operators can achieve a superior denoising performance against CNNs with both
equivalent and well-known deep configurations.
