  Quick and accurate diagnosis is of paramount importance to mitigate the
effects of COVID-19 infection, particularly for severe cases. Enormous effort
has been put towards developing deep learning methods to classify and detect
COVID-19 infections from chest radiography images. However, recently some
questions have been raised surrounding the clinical viability and effectiveness
of such methods. In this work, we investigate the impact of multi-task learning
(classification and segmentation) on the ability of CNNs to differentiate
between various appearances of COVID-19 infections in the lung. We also employ
self-supervised pre-training approaches, namely MoCo and inpainting-CXR, to
eliminate the dependence on expensive ground truth annotations for COVID-19
classification. Finally, we conduct a critical evaluation of the models to
assess their deploy-readiness and provide insights into the difficulties of
fine-grained COVID-19 multi-class classification from chest X-rays.
