  Convolutional Neural Networks have played a significant role in various
medical imaging tasks like classification and segmentation. They provide
state-of-the-art performance compared to classical image processing algorithms.
However, the major downside of these methods is the high computational
complexity, reliance on high-performance hardware like GPUs and the inherent
black-box nature of the model. In this paper, we propose quantised stand-alone
self-attention based models as an alternative to traditional CNNs. In the
proposed class of networks, convolutional layers are replaced with stand-alone
self-attention layers, and the network parameters are quantised after training.
We experimentally validate the performance of our method on classification and
segmentation tasks. We observe a $50-80\%$ reduction in model size, $60-80\%$
lesser number of parameters, $40-85\%$ fewer FLOPs and $65-80\%$ more energy
efficiency during inference on CPUs. The code will be available at \href
{https://github.com/Rakshith2597/Quantised-Self-Attentive-Deep-Neural-Network}{https://github.com/Rakshith2597/Quantised-Self-Attentive-Deep-Neural-Network}.
