  The ReLU activation function (AF) has been extensively applied in deep neural
networks, in particular Convolutional Neural Networks (CNN), for image
classification despite its unresolved dying ReLU problem, which poses
challenges to reliable applications. This issue has obvious important
implications for critical applications, such as those in healthcare. Recent
approaches are just proposing variations of the activation function within the
same unresolved dying ReLU challenge. This contribution reports a different
research direction by investigating the development of an innovative quantum
approach to the ReLU AF that avoids the dying ReLU problem by disruptive
design. The Leaky ReLU was leveraged as a baseline on which the two quantum
principles of entanglement and superposition were applied to derive the
proposed Quantum ReLU (QReLU) and the modified-QReLU (m-QReLU) activation
functions. Both QReLU and m-QReLU are implemented and made freely available in
TensorFlow and Keras. This original approach is effective and validated
extensively in case studies that facilitate the detection of COVID-19 and
Parkinson Disease (PD) from medical images. The two novel AFs were evaluated in
a two-layered CNN against nine ReLU-based AFs on seven benchmark datasets,
including images of spiral drawings taken via graphic tablets from patients
with Parkinson Disease and healthy subjects, and point-of-care ultrasound
images on the lungs of patients with COVID-19, those with pneumonia and healthy
controls. Despite a higher computational cost, results indicated an overall
higher classification accuracy, precision, recall and F1-score brought about by
either quantum AFs on five of the seven bench-mark datasets, thus demonstrating
its potential to be the new benchmark or gold standard AF in CNNs and aid image
classification tasks involved in critical applications, such as medical
diagnoses of COVID-19 and PD.
